Jul 30 22:38:12.666389 ip-10-0-142-225 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 30 22:38:12.666404 ip-10-0-142-225 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 30 22:38:12.666413 ip-10-0-142-225 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 30 22:38:12.666763 ip-10-0-142-225 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 30 22:38:22.793839 ip-10-0-142-225 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 30 22:38:22.793855 ip-10-0-142-225 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot f03eff02ce1146588fcf32f36b56ed25 --
Jul 30 22:40:18.657330 ip-10-0-142-225 systemd[1]: Starting Kubernetes Kubelet...
Jul 30 22:40:19.042647 ip-10-0-142-225 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 22:40:19.042647 ip-10-0-142-225 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 30 22:40:19.042647 ip-10-0-142-225 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 22:40:19.042647 ip-10-0-142-225 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 30 22:40:19.042647 ip-10-0-142-225 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 30 22:40:19.044616 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.044526    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 30 22:40:19.048975 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.048958    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 22:40:19.048975 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.048976    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.048981    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.048987    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.048991    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.048995    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.048997    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049000    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049003    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049005    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049008    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049010    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049013    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049017    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049019    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049022    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049024    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049027    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049029    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049045    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 22:40:19.049062 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049047    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049050    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049052    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049055    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049058    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049061    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049064    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049067    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049069    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049072    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049075    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049079    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049082    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049084    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049087    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049090    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049092    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049095    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049098    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049101    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 22:40:19.049507 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049103    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049106    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049109    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049113    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049116    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049120    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049122    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049125    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049128    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049131    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049133    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049136    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049139    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049141    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049145    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049147    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049151    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049156    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049161    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 22:40:19.049988 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049164    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049167    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049169    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049172    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049175    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049177    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049180    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049183    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049185    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049188    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049190    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049193    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049196    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049199    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049201    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049204    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049207    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049209    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049211    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049214    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 22:40:19.050457 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049216    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049219    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049221    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049224    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049226    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049229    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049231    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049235    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049237    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049240    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049242    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049245    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049248    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049250    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049253    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049255    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049258    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049261    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049263    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049266    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 22:40:19.050919 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049990    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.049997    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050001    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050004    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050007    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050011    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050014    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050017    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050020    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050022    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050024    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050027    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050029    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050047    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050050    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050052    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050055    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050058    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050060    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050065    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 22:40:19.051398 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050068    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050071    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050073    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050076    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050079    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050082    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050084    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050087    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050089    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050092    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050094    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050097    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050099    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050102    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050104    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050107    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050110    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050113    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050116    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050118    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 22:40:19.051908 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050121    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050123    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050125    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050129    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050133    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050136    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050139    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050142    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050144    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050147    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050149    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050152    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050155    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050157    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050160    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050162    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050165    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050167    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050170    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 22:40:19.052426 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050173    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050176    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050178    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050180    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050183    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050186    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050188    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050191    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050193    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050195    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050198    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050200    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050203    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050206    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050208    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050211    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050213    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050216    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050218    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 22:40:19.052874 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050221    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050225    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050228    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050231    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050233    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050236    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050239    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050241    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050244    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050246    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050249    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050251    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050254    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050256    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050259    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050262    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050265    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050268    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050271    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 22:40:19.053357 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050274    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050276    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050652    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050658    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050661    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050664    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050667    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050670    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050673    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050676    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050679    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050682    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050685    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050688    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050691    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050694    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050697    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050700    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050703    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 22:40:19.053820 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050705    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050708    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050711    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050714    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050716    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050719    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050721    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050724    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050726    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050729    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050731    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050734    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050736    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050739    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050741    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050744    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050746    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050748    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050751    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050753    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 22:40:19.054294 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050756    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050760    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050764    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050768    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050771    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050774    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050778    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050781    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050784    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050787    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050790    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050792    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050795    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050798    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050801    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050803    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050806    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050809    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050811    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 22:40:19.054754 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050814    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050816    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050819    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050821    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050824    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050826    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050829    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050831    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050833    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050836    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050839    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050841    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050844    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050846    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050851    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050854    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050856    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050859    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050862    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050865    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 22:40:19.055213 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050867    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050869    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050872    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050877    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050879    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050884    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050888    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050890    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050893    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050895    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050898    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050901    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050903    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050906    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050908    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050911    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050913    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050916    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050918    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050921    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 22:40:19.055691 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050923    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050926    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.050928    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051000    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051010    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051020    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051030    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051052    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051058    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051064    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051070    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051074    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051077    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051081    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051084    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051089    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051092    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051095    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051098    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051101    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051104    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051108    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051111    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051114    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 30 22:40:19.056178 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051118    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051121    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051125    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051128    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051131    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051134    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051137    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051141    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051144    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051147    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051150    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051154    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051157    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051159    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051163    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051166    2578 flags.go:64] FLAG: --enable-server="true"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051170    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051175    2578 flags.go:64] FLAG: --event-burst="100"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051178    2578 flags.go:64] FLAG: --event-qps="50"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051181    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051183    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051186    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051190    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051194    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051197    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 30 22:40:19.056768 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051200    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051203    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051206    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051209    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051212    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051215    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051217    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051220    2578 flags.go:64] FLAG: --feature-gates=""
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051224    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051227    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051230    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051233    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051236    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051239    2578 flags.go:64] FLAG: --help="false"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051242    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051245    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051248    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051250    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051253    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051257    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051259    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051264    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051267    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 30 22:40:19.057379 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051270    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051273    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051276    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051279    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051282    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051284    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051287    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051291    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051294    2578 flags.go:64] FLAG: --lock-file=""
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051297    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051300    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051303    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051309    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051312    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051314    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051317    2578 flags.go:64] FLAG: --logging-format="text"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051320    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051323    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051326    2578 flags.go:64] FLAG: --manifest-url=""
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051329    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051333    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051336    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051340    2578 flags.go:64] FLAG: --max-pods="110"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051343    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051346    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 30 22:40:19.057970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051349    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051352    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051355    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051358    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051361    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051370    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051373    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051376    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051379    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051382    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051388    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051390    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051394    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051396    2578 flags.go:64] FLAG: --port="10250"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051401    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051404    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e898c3232b9a8a94"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051408    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051411    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051414    2578 flags.go:64] FLAG: --register-node="true"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051417    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051420    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051423    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051426    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051428    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051432    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 30 22:40:19.058597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051435    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051438    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051441    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051443    2578 flags.go:64] FLAG: --runonce="false"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051446    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051449    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051452    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051455    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051458    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051461    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051464    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051467    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051469    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051473    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051476    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051479    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051482    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051485    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051488    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051494    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051496    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051500    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051504    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051507    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051510    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 30 22:40:19.059219 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051513    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051516    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051519    2578 flags.go:64] FLAG: --v="2"
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051523    2578 flags.go:64] FLAG: --version="false"
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051526    2578 flags.go:64] FLAG: --vmodule=""
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051530    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.051533    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051647    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051650    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051653    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051656    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051659    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051661    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051664    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051667    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051670    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051672    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051676    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051679    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051682    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051686    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 22:40:19.059823 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051689    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051692    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051694    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051697    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051700    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051703    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051705    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051710    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051713    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051716    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051720    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051723    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051726    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051728    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051731    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051734    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051736    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051739    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051741    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 22:40:19.060462 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051744    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051746    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051749    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051752    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051754    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051757    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051759    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051762    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051764    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051767    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051769    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051772    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051776    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051779    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051782    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051786    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051789    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051792    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051795    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051798    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 22:40:19.060911 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051802    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051804    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051807    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051809    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051812    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051814    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051817    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051820    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051822    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051825    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051828    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051830    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051833    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051835    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051838    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051840    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051843    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051845    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051848    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 22:40:19.061492 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051850    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051853    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051855    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051858    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051860    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051865    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051868    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051870    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051873    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051875    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051878    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051880    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051883    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051887    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051890    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051892    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051894    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051897    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051900    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051902    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 22:40:19.061984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051905    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051907    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051910    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051913    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051915    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051918    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.051921    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.052410    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.058817    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.058832    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.058852    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.058858    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058927    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058932    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058935    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 22:40:19.062514 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058939    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058942    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058946    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058950    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058953    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058956    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058959    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058962    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058965    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058969    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058973    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058975    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058978    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058981    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058983    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058986    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058988    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058991    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058994    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 22:40:19.062918 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.058998    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059002    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059006    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059010    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059014    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059018    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059022    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059026    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059029    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059050    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059053    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059056    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059059    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059061    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059064    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059067    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059070    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059072    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059075    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059078    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 22:40:19.063388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059080    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059083    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059086    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059088    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059092    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059096    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059099    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059102    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059104    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059107    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059110    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059113    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059115    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059118    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059121    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059123    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059126    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059129    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059132    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059134    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 22:40:19.063897 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059138    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059140    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059143    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059145    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059148    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059151    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059153    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059156    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059158    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059161    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059164    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059166    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059169    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059171    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059174    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059176    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059178    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059181    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059184    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 22:40:19.064402 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059186    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059189    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059191    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059194    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059196    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059199    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059201    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059204    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059206    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059209    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059211    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059215    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059218    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059220    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059224    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059227    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059230    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 22:40:19.064873 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059232    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.059237    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059352    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059358    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059361    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059364    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059368    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059370    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059373    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059376    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059379    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059381    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059384    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059387    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059389    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 30 22:40:19.065522 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059392    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059395    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059398    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059400    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059403    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059405    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059408    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059410    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059413    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059416    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059419    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059423    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059426    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059429    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059432    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059435    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059437    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059440    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059442    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 30 22:40:19.065887 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059445    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059447    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059450    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059452    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059455    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059458    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059460    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059462    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059465    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059467    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059470    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059472    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059475    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059477    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059480    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059482    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059485    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059487    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059490    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059492    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 30 22:40:19.066473 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059495    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059497    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059500    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059502    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059505    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059508    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059510    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059513    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059516    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059520    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059523    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059526    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059529    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059532    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059535    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059538    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059540    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059543    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059546    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 30 22:40:19.067115 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059548    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059551    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059554    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059558    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059561    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059564    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059567    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059570    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059572    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059575    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059578    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059581    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059584    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059587    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059589    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059592    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059594    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059597    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059599    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 30 22:40:19.067672 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059602    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059604    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059607    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059609    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059612    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059615    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059617    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059620    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:19.059622    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.059627    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.059745    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.061607    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.062660    2578 server.go:1019] "Starting client certificate rotation"
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.062756    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 22:40:19.068318 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.062823    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 30 22:40:19.068850 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.065074    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 22:40:19.069941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.069921    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 30 22:40:19.084735 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.084719    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 30 22:40:19.086673 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.086657    2578 log.go:25] "Validated CRI v1 image API"
Jul 30 22:40:19.087652 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.087632    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 30 22:40:19.090910 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.090892    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 22:40:19.091781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.091758    2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b755d9ff-cfec-4543-900d-d4a8b1de1c92:/dev/nvme0n1p3 bb5be090-418e-45d2-aca0-abc9cfb73cde:/dev/nvme0n1p4]
Jul 30 22:40:19.091854 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.091779    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 30 22:40:19.096796 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.096674    2578 manager.go:216] Machine: {Timestamp:2026-07-30 22:40:19.09548262 +0000 UTC m=+0.330461292 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100475 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26fb1c319f1a9565ba32ab7401e893 SystemUUID:ec26fb1c-319f-1a95-65ba-32ab7401e893 BootID:f03eff02-ce11-4658-8fcf-32f36b56ed25 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e8:13:1b:db:13 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e8:13:1b:db:13 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0a:17:cf:73:e3:f3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 30 22:40:19.096796 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.096787    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 30 22:40:19.096961 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.096938    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 30 22:40:19.099396 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.099371    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 30 22:40:19.099530 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.099399    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-225.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 30 22:40:19.099999 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.099988    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 30 22:40:19.100054 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.100002    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 30 22:40:19.100054 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.100009    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 22:40:19.100587 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.100576    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 30 22:40:19.101676 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.101666    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 30 22:40:19.104724 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.104712    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 22:40:19.105013 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.105002    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 30 22:40:19.105072 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.105048    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 30 22:40:19.105072 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.105061    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 30 22:40:19.105072 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.105072    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 30 22:40:19.105171 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.105095    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 30 22:40:19.105171 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.105106    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 30 22:40:19.105447 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.105436    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 22:40:19.105486 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.105454    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 30 22:40:19.107222 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.107205    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rlszt"
Jul 30 22:40:19.107684 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.107669    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 30 22:40:19.108911 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.108895    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 30 22:40:19.108955 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.108940    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 30 22:40:19.109492 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109481    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 30 22:40:19.109523 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109496    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 30 22:40:19.109523 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109511    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 30 22:40:19.109523 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109516    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 30 22:40:19.109523 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109524    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 30 22:40:19.109625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109529    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 30 22:40:19.109625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109563    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 30 22:40:19.109625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109569    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 30 22:40:19.109625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109576    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 30 22:40:19.109625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109581    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 30 22:40:19.109625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109593    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 30 22:40:19.109625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.109600    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 30 22:40:19.110320 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.110310    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 30 22:40:19.110350 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.110320    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 30 22:40:19.113785 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.113760    2578 server.go:1268] "Started kubelet"
Jul 30 22:40:19.113972 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.113941    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 30 22:40:19.116076 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.115999    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 30 22:40:19.116217 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.116199    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 30 22:40:19.116621 ip-10-0-142-225 systemd[1]: Started Kubernetes Kubelet.
Jul 30 22:40:19.118694 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.117090    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-225.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 30 22:40:19.118694 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.117162    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-225.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 30 22:40:19.118694 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.117271    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 30 22:40:19.118694 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.117460    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rlszt"
Jul 30 22:40:19.118694 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.117458    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 30 22:40:19.120092 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.120076    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 30 22:40:19.121389 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.120349    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-225.ec2.internal.18c7326c8f0fd822  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-225.ec2.internal,UID:ip-10-0-142-225.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-225.ec2.internal,},FirstTimestamp:2026-07-30 22:40:19.113728034 +0000 UTC m=+0.348706705,LastTimestamp:2026-07-30 22:40:19.113728034 +0000 UTC m=+0.348706705,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-225.ec2.internal,}"
Jul 30 22:40:19.123304 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.123281    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 30 22:40:19.126210 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.126191    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 30 22:40:19.126210 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.126206    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 30 22:40:19.126356 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.126290    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:19.126822 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.126793    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 30 22:40:19.126959 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.126945    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 30 22:40:19.127021 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.126962    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 30 22:40:19.127021 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.126975    2578 factory.go:55] Registering systemd factory
Jul 30 22:40:19.127021 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.126992    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 30 22:40:19.127271 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.127220    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 30 22:40:19.127321 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.127274    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 30 22:40:19.127321 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.127235    2578 factory.go:153] Registering CRI-O factory
Jul 30 22:40:19.127321 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.127295    2578 factory.go:223] Registration of the crio container factory successfully
Jul 30 22:40:19.127449 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.127379    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 30 22:40:19.127449 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.127401    2578 factory.go:103] Registering Raw factory
Jul 30 22:40:19.127449 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.127415    2578 manager.go:1196] Started watching for new ooms in manager
Jul 30 22:40:19.127831 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.127816    2578 manager.go:349] Starting recovery of all containers
Jul 30 22:40:19.137756 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.137732    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 22:40:19.137967 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.137953    2578 manager.go:354] Recovery completed
Jul 30 22:40:19.139327 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.139296    2578 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 30 22:40:19.141552 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.141521    2578 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-225.ec2.internal\" not found" node="ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.142639 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.142627    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 22:40:19.144649 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.144634    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 22:40:19.144714 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.144662    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 22:40:19.144714 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.144672    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID"
Jul 30 22:40:19.145176 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.145164    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 30 22:40:19.145176 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.145172    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 22:40:19.145249 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.145177    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 30 22:40:19.145249 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.145196    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 30 22:40:19.145858 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.145843    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 22:40:19.145922 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.145873    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 22:40:19.145922 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.145883    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID"
Jul 30 22:40:19.146341 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.146329    2578 policy_none.go:49] "None policy: Start"
Jul 30 22:40:19.146375 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.146345    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 30 22:40:19.146375 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.146356    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 30 22:40:19.147148 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.147137    2578 policy_none.go:47] "Start"
Jul 30 22:40:19.188668 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.188650    2578 manager.go:337] "Starting Device Plugin manager"
Jul 30 22:40:19.188788 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.188735    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 30 22:40:19.188788 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.188748    2578 server.go:87] "Starting device plugin registration server"
Jul 30 22:40:19.189018 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.189006    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 30 22:40:19.189116 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.189022    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 30 22:40:19.189170 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.189138    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 30 22:40:19.189235 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.189218    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 30 22:40:19.189235 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.189232    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 30 22:40:19.189734 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.189715    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 30 22:40:19.189809 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.189768    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:19.274235 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.274198    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 30 22:40:19.275557 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.275542    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 30 22:40:19.275625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.275579    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 30 22:40:19.275625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.275603    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 30 22:40:19.275723 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.275638    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 30 22:40:19.277718 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.277700    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 22:40:19.289735 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.289718    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 22:40:19.290551 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.290532    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 22:40:19.290634 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.290568    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 22:40:19.290634 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.290583    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID"
Jul 30 22:40:19.290634 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.290605    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.298929 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.298892    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.298929 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.298913    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-225.ec2.internal\": node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:19.312328 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.312302    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:19.376364 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.376329    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal"]
Jul 30 22:40:19.376521 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.376416    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 22:40:19.378154 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.378134    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 22:40:19.378231 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.378169    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 22:40:19.378231 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.378182    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID"
Jul 30 22:40:19.379621 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.379608    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 22:40:19.379740 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.379724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.379783 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.379770    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 22:40:19.380408 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.380389    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 22:40:19.380494 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.380419    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 22:40:19.380494 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.380424    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 22:40:19.380494 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.380437    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 22:40:19.380494 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.380439    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID"
Jul 30 22:40:19.380494 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.380454    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID"
Jul 30 22:40:19.381597 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.381582    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.381673 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.381608    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 30 22:40:19.382270 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.382257    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientMemory"
Jul 30 22:40:19.382341 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.382281    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasNoDiskPressure"
Jul 30 22:40:19.382341 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.382295    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeHasSufficientPID"
Jul 30 22:40:19.412409 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.412383    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:19.413791 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.413776    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-225.ec2.internal\" not found" node="ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.418399 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.418384    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-225.ec2.internal\" not found" node="ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.429391 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.429373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c4b33d732e720c94e138a741dc2f7ae-config\") pod \"kube-apiserver-proxy-ip-10-0-142-225.ec2.internal\" (UID: \"1c4b33d732e720c94e138a741dc2f7ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.429485 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.429401    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b8f80c89fd4f927b5d7ef81c500684aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"b8f80c89fd4f927b5d7ef81c500684aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.429485 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.429426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8f80c89fd4f927b5d7ef81c500684aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"b8f80c89fd4f927b5d7ef81c500684aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.513430 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.513384    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:19.529747 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.529718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c4b33d732e720c94e138a741dc2f7ae-config\") pod \"kube-apiserver-proxy-ip-10-0-142-225.ec2.internal\" (UID: \"1c4b33d732e720c94e138a741dc2f7ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.529847 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.529766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b8f80c89fd4f927b5d7ef81c500684aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"b8f80c89fd4f927b5d7ef81c500684aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.529847 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.529822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8f80c89fd4f927b5d7ef81c500684aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"b8f80c89fd4f927b5d7ef81c500684aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.529941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.529839    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1c4b33d732e720c94e138a741dc2f7ae-config\") pod \"kube-apiserver-proxy-ip-10-0-142-225.ec2.internal\" (UID: \"1c4b33d732e720c94e138a741dc2f7ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.529941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.529875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8f80c89fd4f927b5d7ef81c500684aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"b8f80c89fd4f927b5d7ef81c500684aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.529941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.529899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b8f80c89fd4f927b5d7ef81c500684aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal\" (UID: \"b8f80c89fd4f927b5d7ef81c500684aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.614251 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.614169    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:19.714698 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.714671    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:19.715789 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.715772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.721230 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:19.721212    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:19.815646 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.815592    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:19.916219 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:19.916190    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:21.076727 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:21.076693    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:21.122205 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.122175    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 30 22:40:21.122804 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.122313    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 22:40:21.122804 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.122346    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 30 22:40:21.177760 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:21.177729    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:21.185034 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.185004    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-29 22:35:20 +0000 UTC" deadline="2028-04-30 08:10:46.6333476 +0000 UTC"
Jul 30 22:40:21.185127 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.185035    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15345h30m25.448316929s"
Jul 30 22:40:21.278825 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:21.278790    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:21.285520 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.285488    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 22:40:21.356064 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:21.356001    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c4b33d732e720c94e138a741dc2f7ae.slice/crio-021a3d8bc9c6cbb7caa824c65cdb85ad3214f06bdf1451c7ee5a30cad0b7fa43 WatchSource:0}: Error finding container 021a3d8bc9c6cbb7caa824c65cdb85ad3214f06bdf1451c7ee5a30cad0b7fa43: Status 404 returned error can't find the container with id 021a3d8bc9c6cbb7caa824c65cdb85ad3214f06bdf1451c7ee5a30cad0b7fa43
Jul 30 22:40:21.358221 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:21.358194    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8f80c89fd4f927b5d7ef81c500684aa.slice/crio-072dd71312b3bc3339a6de023a9ba644f166d6665132d574380d6f12b31d7017 WatchSource:0}: Error finding container 072dd71312b3bc3339a6de023a9ba644f166d6665132d574380d6f12b31d7017: Status 404 returned error can't find the container with id 072dd71312b3bc3339a6de023a9ba644f166d6665132d574380d6f12b31d7017
Jul 30 22:40:21.362924 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.362896    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 22:40:21.379714 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:21.379693    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:21.480291 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:21.480238    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:21.580829 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:21.580747    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-225.ec2.internal\" not found"
Jul 30 22:40:21.613502 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.613468    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 22:40:21.686718 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.686686    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:21.696592 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.696564    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 22:40:21.697488 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.697470    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal"
Jul 30 22:40:21.704901 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:21.704876    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 30 22:40:22.009890 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.009858    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 22:40:22.166539 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.166507    2578 apiserver.go:52] "Watching apiserver"
Jul 30 22:40:22.175864 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.175837    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 30 22:40:22.176357 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.176333    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-tq7vv","openshift-multus/network-metrics-daemon-m7mw5","openshift-ovn-kubernetes/ovnkube-node-xj7kf","kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k","openshift-dns/node-resolver-rn2w4","openshift-image-registry/node-ca-7xbdc","openshift-multus/multus-2tmpp","openshift-network-diagnostics/network-check-target-z2wbw","openshift-network-operator/iptables-alerter-qjbxr","kube-system/global-pull-secret-syncer-989b2","kube-system/konnectivity-agent-wj9cn","openshift-cluster-node-tuning-operator/tuned-6vx87","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal"]
Jul 30 22:40:22.177863 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.177842    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.179002 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.178948    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.180652 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.180630    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-b9l5j\""
Jul 30 22:40:22.180961 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.180940    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 30 22:40:22.181035 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.180943    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 30 22:40:22.181235 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.181215    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 30 22:40:22.181521 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.181503    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.181877 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.181859    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 30 22:40:22.181949 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.181933    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wxtbt\""
Jul 30 22:40:22.182036 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.182020    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 30 22:40:22.182770 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.182543    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.183545 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.183526    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lcqg4\""
Jul 30 22:40:22.183640 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.183619    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 30 22:40:22.183890 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.183870    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.184444 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.184429    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 30 22:40:22.184444 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.184440    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 30 22:40:22.185952 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.185281    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.185952 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.185365    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 30 22:40:22.185952 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.185376    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 30 22:40:22.185952 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.185672    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 30 22:40:22.185952 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.185717    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 30 22:40:22.185952 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.185955    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 30 22:40:22.186560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.185999    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r784g\""
Jul 30 22:40:22.186560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.186021    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 30 22:40:22.186560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.186089    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 30 22:40:22.186560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.186238    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zhwpj\""
Jul 30 22:40:22.186560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.186261    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 30 22:40:22.186560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.186242    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 30 22:40:22.186560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.186421    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 30 22:40:22.186560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.186457    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:22.186560 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.186519    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:22.187029 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.187011    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 30 22:40:22.187771 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.187750    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 30 22:40:22.187860 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.187821    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 30 22:40:22.187994 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.187957    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-f6dsl\""
Jul 30 22:40:22.187994 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.187988    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.189530 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.189496    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:22.189666 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.189569    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:22.190362 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.190323    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 30 22:40:22.190362 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.190337    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j2hkk\""
Jul 30 22:40:22.190566 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.190548    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 30 22:40:22.190654 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.190569    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 30 22:40:22.191583 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.191325    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.192677 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.192656    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.192767 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.192722    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:22.195697 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.195678    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:22.197312 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.196788    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 30 22:40:22.197312 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.196792    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 30 22:40:22.197312 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.196998    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f666g\""
Jul 30 22:40:22.199487 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199041    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dg88w\""
Jul 30 22:40:22.199487 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199202    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 30 22:40:22.199487 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199317    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-var-lib-kubelet\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.199487 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199367    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7cf62e27-4b97-435a-85d0-cb7b09b2040d-ovnkube-script-lib\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.199487 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-os-release\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.199487 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199411    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 30 22:40:22.199781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnwns\" (UniqueName: \"kubernetes.io/projected/d5022176-5f83-4789-8956-88137c655729-kube-api-access-fnwns\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.199781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199512    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 30 22:40:22.199781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-slash\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.199781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-system-cni-dir\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.199781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199585    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-var-lib-kubelet\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.199781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199618    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-lib-modules\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.199781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-cni-bin\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.199781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7cf62e27-4b97-435a-85d0-cb7b09b2040d-ovn-node-metrics-cert\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.199781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199755    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-registration-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.200187 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-modprobe-d\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.200187 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199852    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-kubernetes\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.200187 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-sysctl-d\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.200187 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5022176-5f83-4789-8956-88137c655729-etc-tuned\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.201311 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.199970    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-var-lib-openvswitch\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.201407 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201331    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jc5n\" (UniqueName: \"kubernetes.io/projected/7cf62e27-4b97-435a-85d0-cb7b09b2040d-kube-api-access-8jc5n\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.201407 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201356    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-cni-dir\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.201407 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-sysctl-conf\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.201555 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201428    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsdsx\" (UniqueName: \"kubernetes.io/projected/2cc5821f-7a57-4654-9f65-e309bd1d58b8-kube-api-access-xsdsx\") pod \"node-ca-7xbdc\" (UID: \"2cc5821f-7a57-4654-9f65-e309bd1d58b8\") " pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.201555 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-systemd\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.201555 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201472    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5022176-5f83-4789-8956-88137c655729-tmp\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.201555 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-run-systemd\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.201555 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201514    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-run-k8s-cni-cncf-io\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.201779 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201558    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-kubelet\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.201779 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201581    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-socket-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.201779 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201605    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-run-multus-certs\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.201779 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201629    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cthw6\" (UniqueName: \"kubernetes.io/projected/3d946ef9-4815-4e0f-bd6d-13e28613e05c-kube-api-access-cthw6\") pod \"iptables-alerter-qjbxr\" (UID: \"3d946ef9-4815-4e0f-bd6d-13e28613e05c\") " pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.201779 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-run-netns\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.201779 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.201779 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201736    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-var-lib-cni-multus\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.201779 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201761    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-systemd-units\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201784    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-cni-netd\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201812    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/351a395f-318f-472a-918e-c463a6326d31-tmp-dir\") pod \"node-resolver-rn2w4\" (UID: \"351a395f-318f-472a-918e-c463a6326d31\") " pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b924x\" (UniqueName: \"kubernetes.io/projected/351a395f-318f-472a-918e-c463a6326d31-kube-api-access-b924x\") pod \"node-resolver-rn2w4\" (UID: \"351a395f-318f-472a-918e-c463a6326d31\") " pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201889    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-socket-dir-parent\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201911    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-run-netns\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201932    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-run\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201956    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-run-openvswitch\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.201980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-run-ovn-kubernetes\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202003    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/351a395f-318f-472a-918e-c463a6326d31-hosts-file\") pod \"node-resolver-rn2w4\" (UID: \"351a395f-318f-472a-918e-c463a6326d31\") " pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.202031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-conf-dir\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2cc5821f-7a57-4654-9f65-e309bd1d58b8-serviceca\") pod \"node-ca-7xbdc\" (UID: \"2cc5821f-7a57-4654-9f65-e309bd1d58b8\") " pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202105    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-etc-openvswitch\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202152    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-var-lib-cni-bin\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-cnibin\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202200    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-cni-binary-copy\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202222    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-sys\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202250    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-node-log\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7cf62e27-4b97-435a-85d0-cb7b09b2040d-env-overrides\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202326    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-daemon-config\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202350    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3d946ef9-4815-4e0f-bd6d-13e28613e05c-iptables-alerter-script\") pod \"iptables-alerter-qjbxr\" (UID: \"3d946ef9-4815-4e0f-bd6d-13e28613e05c\") " pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202411    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d946ef9-4815-4e0f-bd6d-13e28613e05c-host-slash\") pod \"iptables-alerter-qjbxr\" (UID: \"3d946ef9-4815-4e0f-bd6d-13e28613e05c\") " pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-run-ovn\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202441    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-device-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.202526 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202454    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-sysconfig\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-sys-fs\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5xff\" (UniqueName: \"kubernetes.io/projected/63f2ff32-b3ab-4325-9807-6fb82e5771c4-kube-api-access-r5xff\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202519    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8n4k\" (UniqueName: \"kubernetes.io/projected/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-kube-api-access-c8n4k\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202541    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2cc5821f-7a57-4654-9f65-e309bd1d58b8-host\") pod \"node-ca-7xbdc\" (UID: \"2cc5821f-7a57-4654-9f65-e309bd1d58b8\") " pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202575    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-host\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202598    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-log-socket\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202621    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7cf62e27-4b97-435a-85d0-cb7b09b2040d-ovnkube-config\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202644    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-hostroot\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.202667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-etc-kubernetes\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.213876 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.213856    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6ckfc"
Jul 30 22:40:22.220664 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.220642    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6ckfc"
Jul 30 22:40:22.288011 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.287934    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 30 22:40:22.303462 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-sysconfig\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.303462 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-sys-fs\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5xff\" (UniqueName: \"kubernetes.io/projected/63f2ff32-b3ab-4325-9807-6fb82e5771c4-kube-api-access-r5xff\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303510    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8n4k\" (UniqueName: \"kubernetes.io/projected/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-kube-api-access-c8n4k\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2cc5821f-7a57-4654-9f65-e309bd1d58b8-host\") pod \"node-ca-7xbdc\" (UID: \"2cc5821f-7a57-4654-9f65-e309bd1d58b8\") " pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/33ceed57-7068-43fe-82bd-ed7a69ce892d-kubelet-config\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303565    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-sysconfig\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-cni-binary-copy\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303565    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-sys-fs\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303607    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-host\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303607    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2cc5821f-7a57-4654-9f65-e309bd1d58b8-host\") pod \"node-ca-7xbdc\" (UID: \"2cc5821f-7a57-4654-9f65-e309bd1d58b8\") " pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303629    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-log-socket\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7cf62e27-4b97-435a-85d0-cb7b09b2040d-ovnkube-config\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-hostroot\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303680    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-host\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-log-socket\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.303692 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303697    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-etc-kubernetes\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-hostroot\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303722    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzsvq\" (UniqueName: \"kubernetes.io/projected/b062ccea-8395-4cf7-8ad8-6fe3011f432b-kube-api-access-tzsvq\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-var-lib-kubelet\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303774    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-etc-kubernetes\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7cf62e27-4b97-435a-85d0-cb7b09b2040d-ovnkube-script-lib\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303824    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-var-lib-kubelet\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303839    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-os-release\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/33ceed57-7068-43fe-82bd-ed7a69ce892d-dbus\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303887    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgks8\" (UniqueName: \"kubernetes.io/projected/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-kube-api-access-bgks8\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnwns\" (UniqueName: \"kubernetes.io/projected/d5022176-5f83-4789-8956-88137c655729-kube-api-access-fnwns\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-os-release\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-slash\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303960    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-system-cni-dir\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.303983    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-var-lib-kubelet\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-lib-modules\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.304399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-cni-bin\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304040    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-slash\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7cf62e27-4b97-435a-85d0-cb7b09b2040d-ovn-node-metrics-cert\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304099    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-system-cni-dir\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304117    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-registration-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304146    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-modprobe-d\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304170    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-kubernetes\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-sysctl-d\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5022176-5f83-4789-8956-88137c655729-etc-tuned\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-cni-bin\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304268    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-lib-modules\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-var-lib-openvswitch\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304288    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-registration-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-var-lib-openvswitch\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304325    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jc5n\" (UniqueName: \"kubernetes.io/projected/7cf62e27-4b97-435a-85d0-cb7b09b2040d-kube-api-access-8jc5n\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-cni-dir\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304376    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-sysctl-conf\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304378    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-modprobe-d\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.305194 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304393    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304405    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7cf62e27-4b97-435a-85d0-cb7b09b2040d-ovnkube-script-lib\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304407    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-sysctl-d\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsdsx\" (UniqueName: \"kubernetes.io/projected/2cc5821f-7a57-4654-9f65-e309bd1d58b8-kube-api-access-xsdsx\") pod \"node-ca-7xbdc\" (UID: \"2cc5821f-7a57-4654-9f65-e309bd1d58b8\") " pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/32e5a3f3-ee4a-4723-ad23-2221405b4470-konnectivity-ca\") pod \"konnectivity-agent-wj9cn\" (UID: \"32e5a3f3-ee4a-4723-ad23-2221405b4470\") " pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304490    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304537    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-cni-dir\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-var-lib-kubelet\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-systemd\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-sysctl-conf\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5022176-5f83-4789-8956-88137c655729-tmp\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304589    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-kubernetes\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-run-systemd\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-etc-systemd\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304640    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-run-k8s-cni-cncf-io\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304672    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-run-k8s-cni-cncf-io\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.306041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304686    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-run-systemd\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-os-release\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-kubelet\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304754    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-socket-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304778    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-run-multus-certs\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304787    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-kubelet\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-cnibin\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cthw6\" (UniqueName: \"kubernetes.io/projected/3d946ef9-4815-4e0f-bd6d-13e28613e05c-kube-api-access-cthw6\") pod \"iptables-alerter-qjbxr\" (UID: \"3d946ef9-4815-4e0f-bd6d-13e28613e05c\") " pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-run-multus-certs\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304853    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-run-netns\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304884    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-socket-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-run-netns\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304908    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304928    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-var-lib-cni-multus\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304968    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-system-cni-dir\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304975    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-var-lib-cni-multus\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.306861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.304997    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-systemd-units\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-cni-netd\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/351a395f-318f-472a-918e-c463a6326d31-tmp-dir\") pod \"node-resolver-rn2w4\" (UID: \"351a395f-318f-472a-918e-c463a6326d31\") " pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305101    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-systemd-units\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b924x\" (UniqueName: \"kubernetes.io/projected/351a395f-318f-472a-918e-c463a6326d31-kube-api-access-b924x\") pod \"node-resolver-rn2w4\" (UID: \"351a395f-318f-472a-918e-c463a6326d31\") " pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7cf62e27-4b97-435a-85d0-cb7b09b2040d-ovnkube-config\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305176    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-cni-netd\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-socket-dir-parent\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-run-netns\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305323    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-socket-dir-parent\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305331    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-run\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305363    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-run-openvswitch\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-run\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305393    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/351a395f-318f-472a-918e-c463a6326d31-tmp-dir\") pod \"node-resolver-rn2w4\" (UID: \"351a395f-318f-472a-918e-c463a6326d31\") " pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305414    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-run-netns\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305392    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-run-ovn-kubernetes\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.307892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305424    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-host-run-ovn-kubernetes\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/351a395f-318f-472a-918e-c463a6326d31-hosts-file\") pod \"node-resolver-rn2w4\" (UID: \"351a395f-318f-472a-918e-c463a6326d31\") " pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-run-openvswitch\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305472    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-conf-dir\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2cc5821f-7a57-4654-9f65-e309bd1d58b8-serviceca\") pod \"node-ca-7xbdc\" (UID: \"2cc5821f-7a57-4654-9f65-e309bd1d58b8\") " pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/351a395f-318f-472a-918e-c463a6326d31-hosts-file\") pod \"node-resolver-rn2w4\" (UID: \"351a395f-318f-472a-918e-c463a6326d31\") " pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305516    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-etc-openvswitch\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305538    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305562    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-var-lib-cni-bin\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-conf-dir\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305573    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-etc-openvswitch\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305587    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305611    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-cnibin\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-host-var-lib-cni-bin\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305634    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-cni-binary-copy\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-sys\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-node-log\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7cf62e27-4b97-435a-85d0-cb7b09b2040d-env-overrides\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.308813 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-cnibin\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305679    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-etc-selinux\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-node-log\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305828    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5022176-5f83-4789-8956-88137c655729-sys\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-daemon-config\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305935    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2cc5821f-7a57-4654-9f65-e309bd1d58b8-serviceca\") pod \"node-ca-7xbdc\" (UID: \"2cc5821f-7a57-4654-9f65-e309bd1d58b8\") " pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.305974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3d946ef9-4815-4e0f-bd6d-13e28613e05c-iptables-alerter-script\") pod \"iptables-alerter-qjbxr\" (UID: \"3d946ef9-4815-4e0f-bd6d-13e28613e05c\") " pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306003    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d946ef9-4815-4e0f-bd6d-13e28613e05c-host-slash\") pod \"iptables-alerter-qjbxr\" (UID: \"3d946ef9-4815-4e0f-bd6d-13e28613e05c\") " pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-run-ovn\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306088    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-device-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/32e5a3f3-ee4a-4723-ad23-2221405b4470-agent-certs\") pod \"konnectivity-agent-wj9cn\" (UID: \"32e5a3f3-ee4a-4723-ad23-2221405b4470\") " pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7cf62e27-4b97-435a-85d0-cb7b09b2040d-run-ovn\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306245    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-cni-binary-copy\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3d946ef9-4815-4e0f-bd6d-13e28613e05c-host-slash\") pod \"iptables-alerter-qjbxr\" (UID: \"3d946ef9-4815-4e0f-bd6d-13e28613e05c\") " pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306247    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7cf62e27-4b97-435a-85d0-cb7b09b2040d-env-overrides\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.309528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/63f2ff32-b3ab-4325-9807-6fb82e5771c4-device-dir\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.310332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306373    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-multus-daemon-config\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.310332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.306696    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3d946ef9-4815-4e0f-bd6d-13e28613e05c-iptables-alerter-script\") pod \"iptables-alerter-qjbxr\" (UID: \"3d946ef9-4815-4e0f-bd6d-13e28613e05c\") " pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.310332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.307679    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d5022176-5f83-4789-8956-88137c655729-etc-tuned\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.310332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.307749    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7cf62e27-4b97-435a-85d0-cb7b09b2040d-ovn-node-metrics-cert\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.310332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.308937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d5022176-5f83-4789-8956-88137c655729-tmp\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.311234 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.311215    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 22:40:22.311234 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.311236    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 22:40:22.311393 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.311245    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f2z2t for pod openshift-network-diagnostics/network-check-target-z2wbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:22.311393 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.311297    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t podName:63ad352e-50fa-49c7-9dfa-dcff5633ab8e nodeName:}" failed. No retries permitted until 2026-07-30 22:40:22.811274908 +0000 UTC m=+2.986238568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f2z2t" (UniqueName: "kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t") pod "network-check-target-z2wbw" (UID: "63ad352e-50fa-49c7-9dfa-dcff5633ab8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:22.312273 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.312217    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnwns\" (UniqueName: \"kubernetes.io/projected/d5022176-5f83-4789-8956-88137c655729-kube-api-access-fnwns\") pod \"tuned-6vx87\" (UID: \"d5022176-5f83-4789-8956-88137c655729\") " pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.312521 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.312503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsdsx\" (UniqueName: \"kubernetes.io/projected/2cc5821f-7a57-4654-9f65-e309bd1d58b8-kube-api-access-xsdsx\") pod \"node-ca-7xbdc\" (UID: \"2cc5821f-7a57-4654-9f65-e309bd1d58b8\") " pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.313532 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.313509    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5xff\" (UniqueName: \"kubernetes.io/projected/63f2ff32-b3ab-4325-9807-6fb82e5771c4-kube-api-access-r5xff\") pod \"aws-ebs-csi-driver-node-c2l9k\" (UID: \"63f2ff32-b3ab-4325-9807-6fb82e5771c4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.313805 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.313783    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8n4k\" (UniqueName: \"kubernetes.io/projected/9a2e2f7c-5e17-4380-bf37-556b6b61eff0-kube-api-access-c8n4k\") pod \"multus-2tmpp\" (UID: \"9a2e2f7c-5e17-4380-bf37-556b6b61eff0\") " pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.314255 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.314236    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jc5n\" (UniqueName: \"kubernetes.io/projected/7cf62e27-4b97-435a-85d0-cb7b09b2040d-kube-api-access-8jc5n\") pod \"ovnkube-node-xj7kf\" (UID: \"7cf62e27-4b97-435a-85d0-cb7b09b2040d\") " pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.314391 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.314369    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cthw6\" (UniqueName: \"kubernetes.io/projected/3d946ef9-4815-4e0f-bd6d-13e28613e05c-kube-api-access-cthw6\") pod \"iptables-alerter-qjbxr\" (UID: \"3d946ef9-4815-4e0f-bd6d-13e28613e05c\") " pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.314524 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.314510    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b924x\" (UniqueName: \"kubernetes.io/projected/351a395f-318f-472a-918e-c463a6326d31-kube-api-access-b924x\") pod \"node-resolver-rn2w4\" (UID: \"351a395f-318f-472a-918e-c463a6326d31\") " pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.333590 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.333561    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 30 22:40:22.339737 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.339685    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" event={"ID":"1c4b33d732e720c94e138a741dc2f7ae","Type":"ContainerStarted","Data":"021a3d8bc9c6cbb7caa824c65cdb85ad3214f06bdf1451c7ee5a30cad0b7fa43"}
Jul 30 22:40:22.340837 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.340814    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" event={"ID":"b8f80c89fd4f927b5d7ef81c500684aa","Type":"ContainerStarted","Data":"072dd71312b3bc3339a6de023a9ba644f166d6665132d574380d6f12b31d7017"}
Jul 30 22:40:22.406499 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/32e5a3f3-ee4a-4723-ad23-2221405b4470-konnectivity-ca\") pod \"konnectivity-agent-wj9cn\" (UID: \"32e5a3f3-ee4a-4723-ad23-2221405b4470\") " pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:22.406678 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406506    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406678 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406535    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406678 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-os-release\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406678 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406591    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-cnibin\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406678 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406620    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-system-cni-dir\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406693    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-system-cni-dir\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-os-release\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406779    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-cnibin\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406803    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.406941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:22.406941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/32e5a3f3-ee4a-4723-ad23-2221405b4470-agent-certs\") pod \"konnectivity-agent-wj9cn\" (UID: \"32e5a3f3-ee4a-4723-ad23-2221405b4470\") " pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:22.406941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406923    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/33ceed57-7068-43fe-82bd-ed7a69ce892d-kubelet-config\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.407327 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-cni-binary-copy\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.407327 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406977    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzsvq\" (UniqueName: \"kubernetes.io/projected/b062ccea-8395-4cf7-8ad8-6fe3011f432b-kube-api-access-tzsvq\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:22.407327 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.406993    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.407327 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.407012    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/33ceed57-7068-43fe-82bd-ed7a69ce892d-dbus\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.407327 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.407037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgks8\" (UniqueName: \"kubernetes.io/projected/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-kube-api-access-bgks8\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.407327 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.407108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/32e5a3f3-ee4a-4723-ad23-2221405b4470-konnectivity-ca\") pod \"konnectivity-agent-wj9cn\" (UID: \"32e5a3f3-ee4a-4723-ad23-2221405b4470\") " pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:22.407327 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.407201    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.407327 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.407305    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:22.407327 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.407310    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.407746 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.407381    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret podName:33ceed57-7068-43fe-82bd-ed7a69ce892d nodeName:}" failed. No retries permitted until 2026-07-30 22:40:22.90736144 +0000 UTC m=+3.082325120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret") pod "global-pull-secret-syncer-989b2" (UID: "33ceed57-7068-43fe-82bd-ed7a69ce892d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:22.407746 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.407483    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/33ceed57-7068-43fe-82bd-ed7a69ce892d-dbus\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.407746 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.407547    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/33ceed57-7068-43fe-82bd-ed7a69ce892d-kubelet-config\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.407746 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.407549    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-cni-binary-copy\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.407746 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.407641    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:22.407746 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.407701    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs podName:b062ccea-8395-4cf7-8ad8-6fe3011f432b nodeName:}" failed. No retries permitted until 2026-07-30 22:40:22.907683465 +0000 UTC m=+3.082647128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs") pod "network-metrics-daemon-m7mw5" (UID: "b062ccea-8395-4cf7-8ad8-6fe3011f432b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:22.409885 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.409864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/32e5a3f3-ee4a-4723-ad23-2221405b4470-agent-certs\") pod \"konnectivity-agent-wj9cn\" (UID: \"32e5a3f3-ee4a-4723-ad23-2221405b4470\") " pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:22.416230 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.416195    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzsvq\" (UniqueName: \"kubernetes.io/projected/b062ccea-8395-4cf7-8ad8-6fe3011f432b-kube-api-access-tzsvq\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:22.416554 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.416532    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgks8\" (UniqueName: \"kubernetes.io/projected/68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a-kube-api-access-bgks8\") pod \"multus-additional-cni-plugins-tq7vv\" (UID: \"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a\") " pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.490961 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.490916    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qjbxr"
Jul 30 22:40:22.500400 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.500010    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6vx87"
Jul 30 22:40:22.507704 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.507675    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:22.515411 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.515390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2tmpp"
Jul 30 22:40:22.523185 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.523164    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k"
Jul 30 22:40:22.530718 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.530699    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rn2w4"
Jul 30 22:40:22.538233 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.538179    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7xbdc"
Jul 30 22:40:22.548812 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.548776    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tq7vv"
Jul 30 22:40:22.554736 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.554700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:22.910511 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.910429    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:22.910511 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.910466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:22.910511 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:22.910490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:22.910737 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.910589    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:22.910737 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.910594    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:22.910737 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.910614    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 22:40:22.910737 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.910636    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 22:40:22.910737 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.910644    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret podName:33ceed57-7068-43fe-82bd-ed7a69ce892d nodeName:}" failed. No retries permitted until 2026-07-30 22:40:23.910630131 +0000 UTC m=+4.085593788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret") pod "global-pull-secret-syncer-989b2" (UID: "33ceed57-7068-43fe-82bd-ed7a69ce892d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:22.910737 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.910657    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs podName:b062ccea-8395-4cf7-8ad8-6fe3011f432b nodeName:}" failed. No retries permitted until 2026-07-30 22:40:23.910651021 +0000 UTC m=+4.085614677 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs") pod "network-metrics-daemon-m7mw5" (UID: "b062ccea-8395-4cf7-8ad8-6fe3011f432b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:22.910737 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.910649    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f2z2t for pod openshift-network-diagnostics/network-check-target-z2wbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:22.910737 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:22.910712    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t podName:63ad352e-50fa-49c7-9dfa-dcff5633ab8e nodeName:}" failed. No retries permitted until 2026-07-30 22:40:23.910696601 +0000 UTC m=+4.085660271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f2z2t" (UniqueName: "kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t") pod "network-check-target-z2wbw" (UID: "63ad352e-50fa-49c7-9dfa-dcff5633ab8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:23.232922 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.232684    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 22:35:22 +0000 UTC" deadline="2028-02-06 06:05:36.073003186 +0000 UTC"
Jul 30 22:40:23.232922 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.232921    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13327h25m12.840088913s"
Jul 30 22:40:23.250922 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:23.250896    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f7ebb1_37ea_4bcc_9f3e_bc65f0c7704a.slice/crio-081a0b74c8d84d36051935e7cd39397b522dde425892adc86be3daa5edbefb46 WatchSource:0}: Error finding container 081a0b74c8d84d36051935e7cd39397b522dde425892adc86be3daa5edbefb46: Status 404 returned error can't find the container with id 081a0b74c8d84d36051935e7cd39397b522dde425892adc86be3daa5edbefb46
Jul 30 22:40:23.251453 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:23.251431    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d946ef9_4815_4e0f_bd6d_13e28613e05c.slice/crio-a2192dd44674b914bd6d8a8b5cc0a1eb93a2392e5dbf69f95b123ca61335950b WatchSource:0}: Error finding container a2192dd44674b914bd6d8a8b5cc0a1eb93a2392e5dbf69f95b123ca61335950b: Status 404 returned error can't find the container with id a2192dd44674b914bd6d8a8b5cc0a1eb93a2392e5dbf69f95b123ca61335950b
Jul 30 22:40:23.252579 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:23.252557    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc5821f_7a57_4654_9f65_e309bd1d58b8.slice/crio-444b89c5dc33cdff8770f5a6175cf7394ef9f695bb96a59d4c70973ada86c92d WatchSource:0}: Error finding container 444b89c5dc33cdff8770f5a6175cf7394ef9f695bb96a59d4c70973ada86c92d: Status 404 returned error can't find the container with id 444b89c5dc33cdff8770f5a6175cf7394ef9f695bb96a59d4c70973ada86c92d
Jul 30 22:40:23.258160 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:23.258135    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod351a395f_318f_472a_918e_c463a6326d31.slice/crio-ef519a2d482ede2fdae2061b7f33bcb045bbe636d6e7b4799b0ed140a0c55597 WatchSource:0}: Error finding container ef519a2d482ede2fdae2061b7f33bcb045bbe636d6e7b4799b0ed140a0c55597: Status 404 returned error can't find the container with id ef519a2d482ede2fdae2061b7f33bcb045bbe636d6e7b4799b0ed140a0c55597
Jul 30 22:40:23.261835 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:40:23.261813    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf62e27_4b97_435a_85d0_cb7b09b2040d.slice/crio-6c8ca280c3eaa3c0ec703f0f20614ff4789a7d4d121bbd719a1821a0888b6440 WatchSource:0}: Error finding container 6c8ca280c3eaa3c0ec703f0f20614ff4789a7d4d121bbd719a1821a0888b6440: Status 404 returned error can't find the container with id 6c8ca280c3eaa3c0ec703f0f20614ff4789a7d4d121bbd719a1821a0888b6440
Jul 30 22:40:23.336774 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.336748    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:23.336930 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.336781    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:23.336930 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.336885    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:23.337042 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.336978    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:23.348199 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.348166    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7xbdc" event={"ID":"2cc5821f-7a57-4654-9f65-e309bd1d58b8","Type":"ContainerStarted","Data":"444b89c5dc33cdff8770f5a6175cf7394ef9f695bb96a59d4c70973ada86c92d"}
Jul 30 22:40:23.349387 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.349364    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq7vv" event={"ID":"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a","Type":"ContainerStarted","Data":"081a0b74c8d84d36051935e7cd39397b522dde425892adc86be3daa5edbefb46"}
Jul 30 22:40:23.352540 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.352515    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wj9cn" event={"ID":"32e5a3f3-ee4a-4723-ad23-2221405b4470","Type":"ContainerStarted","Data":"ec7eaa55628dace79dc5393847bf51554f309ecc003dcf0db524b27851552eff"}
Jul 30 22:40:23.356801 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.356778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6vx87" event={"ID":"d5022176-5f83-4789-8956-88137c655729","Type":"ContainerStarted","Data":"e6468966172cabf1550c3351a3513c1ca837da4172bdff077ca4fb193a45623d"}
Jul 30 22:40:23.360916 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.360893    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerStarted","Data":"6c8ca280c3eaa3c0ec703f0f20614ff4789a7d4d121bbd719a1821a0888b6440"}
Jul 30 22:40:23.361962 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.361937    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qjbxr" event={"ID":"3d946ef9-4815-4e0f-bd6d-13e28613e05c","Type":"ContainerStarted","Data":"a2192dd44674b914bd6d8a8b5cc0a1eb93a2392e5dbf69f95b123ca61335950b"}
Jul 30 22:40:23.365498 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.365474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k" event={"ID":"63f2ff32-b3ab-4325-9807-6fb82e5771c4","Type":"ContainerStarted","Data":"2d335814b7ad7334f696dc3ad18ea21b36c23a39383b551c10a17793bce920c4"}
Jul 30 22:40:23.366607 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.366586    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2tmpp" event={"ID":"9a2e2f7c-5e17-4380-bf37-556b6b61eff0","Type":"ContainerStarted","Data":"7d611397b7f2616bfe4a8cfb0c51f32c6fa4a7382a7225c7393cebe604331d01"}
Jul 30 22:40:23.367726 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.367698    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rn2w4" event={"ID":"351a395f-318f-472a-918e-c463a6326d31","Type":"ContainerStarted","Data":"ef519a2d482ede2fdae2061b7f33bcb045bbe636d6e7b4799b0ed140a0c55597"}
Jul 30 22:40:23.381095 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.381028    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" podStartSLOduration=2.381014114 podStartE2EDuration="2.381014114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:23.380723559 +0000 UTC m=+3.555687238" watchObservedRunningTime="2026-07-30 22:40:23.381014114 +0000 UTC m=+3.555977794"
Jul 30 22:40:23.916826 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.916733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:23.916826 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.916791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:23.917070 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:23.916831    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:23.917070 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.916970    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:23.917070 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.917033    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret podName:33ceed57-7068-43fe-82bd-ed7a69ce892d nodeName:}" failed. No retries permitted until 2026-07-30 22:40:25.917015554 +0000 UTC m=+6.091979217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret") pod "global-pull-secret-syncer-989b2" (UID: "33ceed57-7068-43fe-82bd-ed7a69ce892d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:23.917462 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.917441    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 22:40:23.917462 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.917466    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 22:40:23.917584 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.917478    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f2z2t for pod openshift-network-diagnostics/network-check-target-z2wbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:23.917584 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.917534    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t podName:63ad352e-50fa-49c7-9dfa-dcff5633ab8e nodeName:}" failed. No retries permitted until 2026-07-30 22:40:25.917510136 +0000 UTC m=+6.092473797 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f2z2t" (UniqueName: "kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t") pod "network-check-target-z2wbw" (UID: "63ad352e-50fa-49c7-9dfa-dcff5633ab8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:23.917696 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.917601    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:23.917696 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:23.917633    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs podName:b062ccea-8395-4cf7-8ad8-6fe3011f432b nodeName:}" failed. No retries permitted until 2026-07-30 22:40:25.917622058 +0000 UTC m=+6.092585718 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs") pod "network-metrics-daemon-m7mw5" (UID: "b062ccea-8395-4cf7-8ad8-6fe3011f432b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:24.234042 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:24.233980    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-29 22:35:22 +0000 UTC" deadline="2028-05-14 12:15:02.937473026 +0000 UTC"
Jul 30 22:40:24.234042 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:24.234041    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15685h34m38.703436303s"
Jul 30 22:40:24.341122 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:24.340587    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:24.341122 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:24.340734    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:24.380115 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:24.378548    2578 generic.go:356] "Generic (PLEG): container finished" podID="b8f80c89fd4f927b5d7ef81c500684aa" containerID="67e03a893906372ee2334bdbe62481318f3c94fc5ac18b74033c61c050c432a8" exitCode=0
Jul 30 22:40:24.380115 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:24.378641    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" event={"ID":"b8f80c89fd4f927b5d7ef81c500684aa","Type":"ContainerDied","Data":"67e03a893906372ee2334bdbe62481318f3c94fc5ac18b74033c61c050c432a8"}
Jul 30 22:40:24.403490 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:24.403452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-225.ec2.internal" event={"ID":"1c4b33d732e720c94e138a741dc2f7ae","Type":"ContainerStarted","Data":"a23023279b3c12b833e5c58f167ab0af622d9d35115ac9d83b1da151afc89f03"}
Jul 30 22:40:25.336182 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:25.336149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:25.336650 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.336285    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:25.336650 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:25.336341    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:25.336650 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.336404    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:25.411705 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:25.411668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" event={"ID":"b8f80c89fd4f927b5d7ef81c500684aa","Type":"ContainerStarted","Data":"6fafd2c403e46900c9d6525d5c503b3e1cbec1503d659f120f0de7fae6e850ae"}
Jul 30 22:40:25.933302 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:25.933262    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:25.933495 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:25.933320    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:25.933495 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:25.933362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:25.933610 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.933541    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:25.933610 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.933607    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret podName:33ceed57-7068-43fe-82bd-ed7a69ce892d nodeName:}" failed. No retries permitted until 2026-07-30 22:40:29.933587167 +0000 UTC m=+10.108550842 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret") pod "global-pull-secret-syncer-989b2" (UID: "33ceed57-7068-43fe-82bd-ed7a69ce892d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:25.934023 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.934000    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 22:40:25.934172 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.934031    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 22:40:25.934172 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.934043    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f2z2t for pod openshift-network-diagnostics/network-check-target-z2wbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:25.934172 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.934104    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t podName:63ad352e-50fa-49c7-9dfa-dcff5633ab8e nodeName:}" failed. No retries permitted until 2026-07-30 22:40:29.934089648 +0000 UTC m=+10.109053308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f2z2t" (UniqueName: "kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t") pod "network-check-target-z2wbw" (UID: "63ad352e-50fa-49c7-9dfa-dcff5633ab8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:25.934172 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.934170    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:25.934391 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:25.934204    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs podName:b062ccea-8395-4cf7-8ad8-6fe3011f432b nodeName:}" failed. No retries permitted until 2026-07-30 22:40:29.934192514 +0000 UTC m=+10.109156173 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs") pod "network-metrics-daemon-m7mw5" (UID: "b062ccea-8395-4cf7-8ad8-6fe3011f432b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:26.338724 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:26.336842    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:26.338724 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:26.336983    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:27.336552 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:27.336514    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:27.336744 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:27.336642    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:27.336941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:27.336921    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:27.337011 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:27.336988    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:28.338871 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:28.338453    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:28.338871 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:28.338571    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:29.336429 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:29.336390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:29.336429 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:29.336411    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:29.336670 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.336531    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:29.336745 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.336714    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:29.968150 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:29.968104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:29.968150 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:29.968157    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:29.968598 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:29.968197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:29.968598 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.968337    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:29.968598 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.968400    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret podName:33ceed57-7068-43fe-82bd-ed7a69ce892d nodeName:}" failed. No retries permitted until 2026-07-30 22:40:37.968380334 +0000 UTC m=+18.143343991 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret") pod "global-pull-secret-syncer-989b2" (UID: "33ceed57-7068-43fe-82bd-ed7a69ce892d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:29.968759 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.968743    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 22:40:29.968807 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.968765    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 22:40:29.968807 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.968775    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f2z2t for pod openshift-network-diagnostics/network-check-target-z2wbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:29.968866 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.968810    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t podName:63ad352e-50fa-49c7-9dfa-dcff5633ab8e nodeName:}" failed. No retries permitted until 2026-07-30 22:40:37.968796939 +0000 UTC m=+18.143760596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f2z2t" (UniqueName: "kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t") pod "network-check-target-z2wbw" (UID: "63ad352e-50fa-49c7-9dfa-dcff5633ab8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:29.968909 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.968865    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:29.968909 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:29.968898    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs podName:b062ccea-8395-4cf7-8ad8-6fe3011f432b nodeName:}" failed. No retries permitted until 2026-07-30 22:40:37.968886805 +0000 UTC m=+18.143850466 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs") pod "network-metrics-daemon-m7mw5" (UID: "b062ccea-8395-4cf7-8ad8-6fe3011f432b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:30.336955 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:30.336873    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:30.337143 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:30.336972    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:31.336565 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:31.336531    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:31.336978 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:31.336681    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:31.336978 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:31.336543    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:31.336978 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:31.336867    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:32.338639 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:32.338609    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:32.339001 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:32.338714    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:33.336300 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:33.336267    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:33.336489 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:33.336267    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:33.336489 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:33.336391    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:33.336489 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:33.336475    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:34.336914 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:34.336869    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:34.337343 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:34.337010    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:35.336553 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:35.336509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:35.336727 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:35.336635    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:35.336727 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:35.336667    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:35.336727 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:35.336722    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:36.336971 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:36.336916    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:36.337375 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:36.337122    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:37.336522 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:37.336478    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:37.336708 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:37.336492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:37.336708 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:37.336603    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:37.336809 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:37.336701    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:38.029592 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:38.029555    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:38.029592 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:38.029592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:38.030082 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:38.029619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:38.030082 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:38.029721    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:38.030082 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:38.029722    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:38.030082 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:38.029739    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 22:40:38.030082 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:38.029763    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 22:40:38.030082 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:38.029777    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f2z2t for pod openshift-network-diagnostics/network-check-target-z2wbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:38.030082 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:38.029783    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret podName:33ceed57-7068-43fe-82bd-ed7a69ce892d nodeName:}" failed. No retries permitted until 2026-07-30 22:40:54.029764862 +0000 UTC m=+34.204728524 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret") pod "global-pull-secret-syncer-989b2" (UID: "33ceed57-7068-43fe-82bd-ed7a69ce892d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:38.030082 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:38.029797    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs podName:b062ccea-8395-4cf7-8ad8-6fe3011f432b nodeName:}" failed. No retries permitted until 2026-07-30 22:40:54.029791093 +0000 UTC m=+34.204754749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs") pod "network-metrics-daemon-m7mw5" (UID: "b062ccea-8395-4cf7-8ad8-6fe3011f432b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:38.030082 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:38.029825    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t podName:63ad352e-50fa-49c7-9dfa-dcff5633ab8e nodeName:}" failed. No retries permitted until 2026-07-30 22:40:54.029808897 +0000 UTC m=+34.204772558 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f2z2t" (UniqueName: "kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t") pod "network-check-target-z2wbw" (UID: "63ad352e-50fa-49c7-9dfa-dcff5633ab8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:38.337089 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:38.336791    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:38.337268 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:38.337164    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:39.336199 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:39.336163    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:39.336199 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:39.336191    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:39.336655 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:39.336282    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:39.336655 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:39.336394    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:40.336911 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:40.336877    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:40.337358 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:40.336995    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:41.336481 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:41.336454    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:41.336639 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:41.336454    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:41.336639 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:41.336552    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:41.336704 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:41.336644    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:42.336856 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.336725    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:42.337503 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:42.336970    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:42.446976 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.446941    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6vx87" event={"ID":"d5022176-5f83-4789-8956-88137c655729","Type":"ContainerStarted","Data":"893ffcf017a83b3a16c548ba5a76ee6d4d1e9ddb720ff2bc57e902645fe4d609"}
Jul 30 22:40:42.448746 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.448726    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:40:42.449081 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.449039    2578 generic.go:356] "Generic (PLEG): container finished" podID="7cf62e27-4b97-435a-85d0-cb7b09b2040d" containerID="1cb69c02589b37d085905bd5c6e8adeef842977c2a1ec9a79ae4f36aeedb968f" exitCode=1
Jul 30 22:40:42.449169 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.449081    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerStarted","Data":"550ecc3d712f4aea684394ceb6a436a396ad244865042a55215b4365fbacb52a"}
Jul 30 22:40:42.449169 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.449147    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerDied","Data":"1cb69c02589b37d085905bd5c6e8adeef842977c2a1ec9a79ae4f36aeedb968f"}
Jul 30 22:40:42.449249 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.449184    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerStarted","Data":"111d64def944ff0789bd49f1c0ab14c044269c7b4eead20eabaa79f40cae52a4"}
Jul 30 22:40:42.450531 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.450510    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k" event={"ID":"63f2ff32-b3ab-4325-9807-6fb82e5771c4","Type":"ContainerStarted","Data":"0a60a36a624d97a18c05b85a3f3a5e0a0e3c3d3b63c8f02182f531b768e8d381"}
Jul 30 22:40:42.451932 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.451906    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2tmpp" event={"ID":"9a2e2f7c-5e17-4380-bf37-556b6b61eff0","Type":"ContainerStarted","Data":"bf4ac96e2179e29ad6783ea7da48433593f97c318f8866ae287ca66e8ac07ba6"}
Jul 30 22:40:42.453414 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.453385    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rn2w4" event={"ID":"351a395f-318f-472a-918e-c463a6326d31","Type":"ContainerStarted","Data":"7903ec6abc0c6890de597dee808c777f62078306171e06c2a588c532838dcf86"}
Jul 30 22:40:42.454812 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.454791    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7xbdc" event={"ID":"2cc5821f-7a57-4654-9f65-e309bd1d58b8","Type":"ContainerStarted","Data":"5ede77e93a25ad3717ae7f9f81087179739e2c09e1b637c3be030af7bf345e7c"}
Jul 30 22:40:42.459916 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.459890    2578 generic.go:356] "Generic (PLEG): container finished" podID="68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a" containerID="7118126a797fd979c0bbb5d8c39b72d87ebe170c1573d560da0b04e29fbf3f62" exitCode=0
Jul 30 22:40:42.460012 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.459964    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq7vv" event={"ID":"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a","Type":"ContainerDied","Data":"7118126a797fd979c0bbb5d8c39b72d87ebe170c1573d560da0b04e29fbf3f62"}
Jul 30 22:40:42.461467 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.461430    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wj9cn" event={"ID":"32e5a3f3-ee4a-4723-ad23-2221405b4470","Type":"ContainerStarted","Data":"40416abb73df513b631ef66ce8a394279d90e49aee74471bab23cdff364c797f"}
Jul 30 22:40:42.462135 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.462094    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-225.ec2.internal" podStartSLOduration=21.462066403 podStartE2EDuration="21.462066403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:25.428784156 +0000 UTC m=+5.603747818" watchObservedRunningTime="2026-07-30 22:40:42.462066403 +0000 UTC m=+22.637030077"
Jul 30 22:40:42.462821 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.462792    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6vx87" podStartSLOduration=3.999769657 podStartE2EDuration="22.462780824s" totalImagesPullingTime="18.463011167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:42.462235734 +0000 UTC m=+22.637199414" watchObservedRunningTime="2026-07-30 22:40:42.462780824 +0000 UTC m=+22.637744503"
Jul 30 22:40:42.483239 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.483196    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2tmpp" podStartSLOduration=4.00058621 podStartE2EDuration="22.483180775s" totalImagesPullingTime="18.482594565s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:42.482708735 +0000 UTC m=+22.657672413" watchObservedRunningTime="2026-07-30 22:40:42.483180775 +0000 UTC m=+22.658144453"
Jul 30 22:40:42.541256 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.532453    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rn2w4" podStartSLOduration=4.063042892 podStartE2EDuration="22.532437611s" totalImagesPullingTime="18.469394719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:42.51386532 +0000 UTC m=+22.688828999" watchObservedRunningTime="2026-07-30 22:40:42.532437611 +0000 UTC m=+22.707401292"
Jul 30 22:40:42.541256 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.532531    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7xbdc" podStartSLOduration=4.056468509 podStartE2EDuration="22.532525783s" totalImagesPullingTime="18.476057274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:42.531962019 +0000 UTC m=+22.706925699" watchObservedRunningTime="2026-07-30 22:40:42.532525783 +0000 UTC m=+22.707489461"
Jul 30 22:40:42.725199 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.725159    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:42.726022 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.726001    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:42.744869 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:42.744817    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wj9cn" podStartSLOduration=12.406662633 podStartE2EDuration="22.744800702s" totalImagesPullingTime="10.338138069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:42.577746732 +0000 UTC m=+22.752710411" watchObservedRunningTime="2026-07-30 22:40:42.744800702 +0000 UTC m=+22.919764375"
Jul 30 22:40:43.336927 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.336726    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:43.337320 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.336739    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:43.337320 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:43.337038    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:43.337320 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:43.337123    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:43.371416 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.371388    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 30 22:40:43.465520 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.465447    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k" event={"ID":"63f2ff32-b3ab-4325-9807-6fb82e5771c4","Type":"ContainerStarted","Data":"cfea6057c1f321010a7ca081334e01cdbd0704556e80704ee2cc8cde793fc26a"}
Jul 30 22:40:43.467991 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.467972    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:40:43.468372 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.468352    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerStarted","Data":"514821df076f43e49a44211028805589879f3367c31b3b68da16c1eaf282210e"}
Jul 30 22:40:43.468446 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.468427    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerStarted","Data":"8845832d42eac8d7e9e8a63f9c24511e103e089c220bec43cce1984ba662ed9d"}
Jul 30 22:40:43.468503 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.468479    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerStarted","Data":"a5d08d175098c318bca2ae127d6a905b3ff56feefee18133f929a01706d22c8a"}
Jul 30 22:40:43.469705 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.469683    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qjbxr" event={"ID":"3d946ef9-4815-4e0f-bd6d-13e28613e05c","Type":"ContainerStarted","Data":"961d7d514890e7c6a9f1ed85cad174463fde72a73c51a712049f747bcb83d065"}
Jul 30 22:40:43.650557 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.650524    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:43.651146 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.651122    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wj9cn"
Jul 30 22:40:43.666018 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:43.665973    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qjbxr" podStartSLOduration=5.189822725 podStartE2EDuration="23.665958413s" totalImagesPullingTime="18.476135688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:43.484876705 +0000 UTC m=+23.659840386" watchObservedRunningTime="2026-07-30 22:40:43.665958413 +0000 UTC m=+23.840922091"
Jul 30 22:40:44.263247 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:44.263132    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-30T22:40:43.371409894Z","UUID":"e280ed0e-f3ad-4d55-9e48-07f75dedd338","Handler":null,"Name":"","Endpoint":""}
Jul 30 22:40:44.265009 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:44.264984    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 30 22:40:44.265210 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:44.265029    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 30 22:40:44.336669 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:44.336639    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:44.336810 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:44.336781    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:45.336310 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:45.336271    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:45.336876 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:45.336275    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:45.336876 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:45.336407    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:45.336876 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:45.336487    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:45.477193 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:45.477164    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:40:45.477544 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:45.477522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerStarted","Data":"c15d43411da55599e9e52574bd19b155188d0c04e1a7da564351c76e2afa7ee8"}
Jul 30 22:40:45.479276 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:45.479247    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k" event={"ID":"63f2ff32-b3ab-4325-9807-6fb82e5771c4","Type":"ContainerStarted","Data":"67c531a25b8a0dc74eca423bf07131beff60d95aa0a0c06a2ed7825ad54d2054"}
Jul 30 22:40:45.495916 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:45.495876    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2l9k" podStartSLOduration=4.87933179 podStartE2EDuration="25.495860462s" totalImagesPullingTime="20.616528672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:45.495796542 +0000 UTC m=+25.670760262" watchObservedRunningTime="2026-07-30 22:40:45.495860462 +0000 UTC m=+25.670824141"
Jul 30 22:40:46.336159 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:46.336125    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:46.336325 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:46.336265    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:47.335805 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:47.335774    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:47.336485 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:47.335774    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:47.336485 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:47.335891    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:47.336485 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:47.335967    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:48.336223 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.335907    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:48.336643 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:48.336297    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:48.487126 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.487091    2578 generic.go:356] "Generic (PLEG): container finished" podID="68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a" containerID="7d058f56eeff9ab3220e52fcacbc216ebe3e3fd84fa12fd4c4293f923b333826" exitCode=0
Jul 30 22:40:48.487308 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.487179    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq7vv" event={"ID":"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a","Type":"ContainerDied","Data":"7d058f56eeff9ab3220e52fcacbc216ebe3e3fd84fa12fd4c4293f923b333826"}
Jul 30 22:40:48.494154 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.494026    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:40:48.494463 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.494442    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerStarted","Data":"f28a538fe38461f95ffd18f9d94236190a02ae39d9fe262c47ed66108fce1db1"}
Jul 30 22:40:48.494767 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.494753    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:48.494868 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.494810    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:48.494868 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.494854    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:48.494961 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.494946    2578 scope.go:117] "RemoveContainer" containerID="1cb69c02589b37d085905bd5c6e8adeef842977c2a1ec9a79ae4f36aeedb968f"
Jul 30 22:40:48.513482 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.513446    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:48.513562 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:48.513551    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:40:49.336016 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.335978    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:49.336224 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.335978    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:49.336224 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:49.336135    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:49.336224 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:49.336189    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:49.487388 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.487362    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m7mw5"]
Jul 30 22:40:49.487543 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.487509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:49.487655 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:49.487626    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:49.490322 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.490297    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z2wbw"]
Jul 30 22:40:49.490909 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.490890    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-989b2"]
Jul 30 22:40:49.498964 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.498935    2578 generic.go:356] "Generic (PLEG): container finished" podID="68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a" containerID="9a7baf2e47dbbf5f7410065b7f6d70d04ae04a0374df9ef42b399508c2e69977" exitCode=0
Jul 30 22:40:49.499126 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.498967    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq7vv" event={"ID":"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a","Type":"ContainerDied","Data":"9a7baf2e47dbbf5f7410065b7f6d70d04ae04a0374df9ef42b399508c2e69977"}
Jul 30 22:40:49.503523 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.503500    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:40:49.503902 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.503872    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" event={"ID":"7cf62e27-4b97-435a-85d0-cb7b09b2040d","Type":"ContainerStarted","Data":"603c1bb6f86566c9939032a864e57099a78963e68787586da38581e086b9eeb0"}
Jul 30 22:40:49.504030 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.503914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:49.504030 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.503891    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:49.507518 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:49.504441    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:49.507518 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:49.504480    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:49.548706 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:49.548655    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf" podStartSLOduration=11.028368059 podStartE2EDuration="29.548638329s" totalImagesPullingTime="18.52027027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:49.546969533 +0000 UTC m=+29.721933225" watchObservedRunningTime="2026-07-30 22:40:49.548638329 +0000 UTC m=+29.723601985"
Jul 30 22:40:50.507955 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:50.507924    2578 generic.go:356] "Generic (PLEG): container finished" podID="68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a" containerID="c41f1422681022c93544c1f1acecf830f4cfb9a25b75daa8e612170469d49220" exitCode=0
Jul 30 22:40:50.508423 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:50.508020    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq7vv" event={"ID":"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a","Type":"ContainerDied","Data":"c41f1422681022c93544c1f1acecf830f4cfb9a25b75daa8e612170469d49220"}
Jul 30 22:40:51.336926 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:51.336851    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:51.337060 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:51.336867    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:51.337060 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:51.336974    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:51.337060 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:51.336971    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:51.337170 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:51.337037    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:51.337170 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:51.337141    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:52.165847 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:52.165818    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rn2w4_351a395f-318f-472a-918e-c463a6326d31/dns-node-resolver/0.log"
Jul 30 22:40:52.745697 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:52.745665    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7xbdc_2cc5821f-7a57-4654-9f65-e309bd1d58b8/node-ca/0.log"
Jul 30 22:40:53.336732 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:53.336528    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:53.337214 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:53.336552    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:53.337214 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:53.336827    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-989b2" podUID="33ceed57-7068-43fe-82bd-ed7a69ce892d"
Jul 30 22:40:53.337214 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:53.336563    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:53.337214 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:53.336911    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wbw" podUID="63ad352e-50fa-49c7-9dfa-dcff5633ab8e"
Jul 30 22:40:53.337214 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:53.337006    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m7mw5" podUID="b062ccea-8395-4cf7-8ad8-6fe3011f432b"
Jul 30 22:40:54.042289 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:54.042249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:54.042501 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:54.042297    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:54.042501 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:54.042328    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:54.042501 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:54.042431    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:54.042501 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:54.042445    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 30 22:40:54.042501 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:54.042464    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:54.042501 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:54.042470    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 30 22:40:54.042501 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:54.042485    2578 projected.go:196] Error preparing data for projected volume kube-api-access-f2z2t for pod openshift-network-diagnostics/network-check-target-z2wbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:54.042501 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:54.042489    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret podName:33ceed57-7068-43fe-82bd-ed7a69ce892d nodeName:}" failed. No retries permitted until 2026-07-30 22:41:26.042473499 +0000 UTC m=+66.217437170 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret") pod "global-pull-secret-syncer-989b2" (UID: "33ceed57-7068-43fe-82bd-ed7a69ce892d") : object "kube-system"/"original-pull-secret" not registered
Jul 30 22:40:54.042905 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:54.042530    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs podName:b062ccea-8395-4cf7-8ad8-6fe3011f432b nodeName:}" failed. No retries permitted until 2026-07-30 22:41:26.042513671 +0000 UTC m=+66.217477344 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs") pod "network-metrics-daemon-m7mw5" (UID: "b062ccea-8395-4cf7-8ad8-6fe3011f432b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 30 22:40:54.042905 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:40:54.042549    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t podName:63ad352e-50fa-49c7-9dfa-dcff5633ab8e nodeName:}" failed. No retries permitted until 2026-07-30 22:41:26.042539061 +0000 UTC m=+66.217502720 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-f2z2t" (UniqueName: "kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t") pod "network-check-target-z2wbw" (UID: "63ad352e-50fa-49c7-9dfa-dcff5633ab8e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 30 22:40:54.659835 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:54.659757    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-225.ec2.internal" event="NodeReady"
Jul 30 22:40:54.660366 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:54.659930    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 30 22:40:55.336291 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.336260    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:40:55.336438 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.336264    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:40:55.336438 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.336416    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:40:55.338986 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.338959    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 22:40:55.340172 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.340143    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzp2k\""
Jul 30 22:40:55.340172 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.340155    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 22:40:55.340172 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.340172    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-29hrf\""
Jul 30 22:40:55.340392 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.340216    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 22:40:55.340392 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.340153    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 22:40:55.522543 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.522513    2578 generic.go:356] "Generic (PLEG): container finished" podID="68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a" containerID="6597201a6b6d5b9ca2aa3f5ab9bf7d985c2dfac7084bd7160084ebd2e7306e90" exitCode=0
Jul 30 22:40:55.522710 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:55.522565    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq7vv" event={"ID":"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a","Type":"ContainerDied","Data":"6597201a6b6d5b9ca2aa3f5ab9bf7d985c2dfac7084bd7160084ebd2e7306e90"}
Jul 30 22:40:56.528223 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:56.528190    2578 generic.go:356] "Generic (PLEG): container finished" podID="68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a" containerID="7eefcb6239f177a6046a596952cf30385d80d7ef3a02233b398b090232bba318" exitCode=0
Jul 30 22:40:56.528223 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:56.528226    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq7vv" event={"ID":"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a","Type":"ContainerDied","Data":"7eefcb6239f177a6046a596952cf30385d80d7ef3a02233b398b090232bba318"}
Jul 30 22:40:57.533501 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:57.533463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tq7vv" event={"ID":"68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a","Type":"ContainerStarted","Data":"0b266c3a9cac23091d8be13d7a01c195877383d6cc041d8d78379c81c673e32f"}
Jul 30 22:40:57.557739 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:40:57.557671    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tq7vv" podStartSLOduration=8.220740829 podStartE2EDuration="37.557654874s" totalImagesPullingTime="29.336914045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:40:57.557489483 +0000 UTC m=+37.732453163" watchObservedRunningTime="2026-07-30 22:40:57.557654874 +0000 UTC m=+37.732618549"
Jul 30 22:41:15.016644 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.016610    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jzsrh"]
Jul 30 22:41:15.018412 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.018396    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jzsrh"
Jul 30 22:41:15.023908 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.023887    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-npr54\""
Jul 30 22:41:15.024263 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.024229    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 30 22:41:15.024424 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.024404    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 30 22:41:15.026646 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.026628    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 30 22:41:15.038829 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.038807    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jzsrh"]
Jul 30 22:41:15.058532 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.058506    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rwrlr"]
Jul 30 22:41:15.060469 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.060450    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.063074 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.063033    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 30 22:41:15.066552 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.066535    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9dd4c\""
Jul 30 22:41:15.066793 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.066776    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 30 22:41:15.066947 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.066929    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 30 22:41:15.066985 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.066932    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 30 22:41:15.093623 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.093598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rwrlr"]
Jul 30 22:41:15.127080 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.127028    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tm4q5"]
Jul 30 22:41:15.129335 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.129312    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"]
Jul 30 22:41:15.129471 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.129435    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.130877 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.130858    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"]
Jul 30 22:41:15.131024 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.131007    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.132496 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.132472    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6758c75797-xh58q"]
Jul 30 22:41:15.132599 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.132575    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"
Jul 30 22:41:15.134148 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.134130    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.143922 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.143805    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 30 22:41:15.143922 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.143815    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 30 22:41:15.144169 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.144154    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 30 22:41:15.144214 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.144206    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 30 22:41:15.144258 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.144242    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 30 22:41:15.144448 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.144433    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 30 22:41:15.144557 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.144543    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 30 22:41:15.144622 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.144609    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 30 22:41:15.149490 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.149473    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-n75j6\""
Jul 30 22:41:15.152415 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.152391    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2cqmw\""
Jul 30 22:41:15.161814 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.161793    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 30 22:41:15.161814 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.161814    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 30 22:41:15.162019 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.162009    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 30 22:41:15.162366 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.162331    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 30 22:41:15.163836 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.163810    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 30 22:41:15.165653 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.165632    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f59gl\""
Jul 30 22:41:15.169079 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.169041    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 30 22:41:15.171144 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.171119    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6758c75797-xh58q"]
Jul 30 22:41:15.177879 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.177857    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tm4q5"]
Jul 30 22:41:15.178836 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.178814    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"]
Jul 30 22:41:15.179510 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.179488    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"]
Jul 30 22:41:15.190463 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.190440    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c82b749b-fe1a-4132-81ec-042cad024f81-crio-socket\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.190560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.190489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2j7vv\" (UniqueName: \"kubernetes.io/projected/c82b749b-fe1a-4132-81ec-042cad024f81-kube-api-access-2j7vv\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.190560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.190508    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f931cf0f-d228-45b8-999a-171024393c99-cert\") pod \"ingress-canary-jzsrh\" (UID: \"f931cf0f-d228-45b8-999a-171024393c99\") " pod="openshift-ingress-canary/ingress-canary-jzsrh"
Jul 30 22:41:15.190560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.190543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szn49\" (UniqueName: \"kubernetes.io/projected/f931cf0f-d228-45b8-999a-171024393c99-kube-api-access-szn49\") pod \"ingress-canary-jzsrh\" (UID: \"f931cf0f-d228-45b8-999a-171024393c99\") " pod="openshift-ingress-canary/ingress-canary-jzsrh"
Jul 30 22:41:15.190657 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.190574    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c82b749b-fe1a-4132-81ec-042cad024f81-data-volume\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.190657 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.190589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c82b749b-fe1a-4132-81ec-042cad024f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.190657 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.190613    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c82b749b-fe1a-4132-81ec-042cad024f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.291852 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.291764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c82b749b-fe1a-4132-81ec-042cad024f81-data-volume\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.291852 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.291798    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c82b749b-fe1a-4132-81ec-042cad024f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.291852 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.291822    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7n2q\" (UniqueName: \"kubernetes.io/projected/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-kube-api-access-c7n2q\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.292141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.291927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c025bbe-ee0c-428a-841f-98c633069cbb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68f54f5f8-4s6wn\" (UID: \"5c025bbe-ee0c-428a-841f-98c633069cbb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"
Jul 30 22:41:15.292141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.291953    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5k9b\" (UniqueName: \"kubernetes.io/projected/9007b6e3-863d-426c-b9c7-c2c3637861af-kube-api-access-g5k9b\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.292141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.291985    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-installation-pull-secrets\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.292141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292014    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s795t\" (UniqueName: \"kubernetes.io/projected/46efc957-0cdf-4401-a831-0820448ca283-kube-api-access-s795t\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.292141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c82b749b-fe1a-4132-81ec-042cad024f81-crio-socket\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.292141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292087    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-ca\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.292141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9007b6e3-863d-426c-b9c7-c2c3637861af-tmp-dir\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.292141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292126    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.292141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-bound-sa-token\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c82b749b-fe1a-4132-81ec-042cad024f81-data-volume\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292168    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-trusted-ca\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f931cf0f-d228-45b8-999a-171024393c99-cert\") pod \"ingress-canary-jzsrh\" (UID: \"f931cf0f-d228-45b8-999a-171024393c99\") " pod="openshift-ingress-canary/ingress-canary-jzsrh"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szn49\" (UniqueName: \"kubernetes.io/projected/f931cf0f-d228-45b8-999a-171024393c99-kube-api-access-szn49\") pod \"ingress-canary-jzsrh\" (UID: \"f931cf0f-d228-45b8-999a-171024393c99\") " pod="openshift-ingress-canary/ingress-canary-jzsrh"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c82b749b-fe1a-4132-81ec-042cad024f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292265    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c82b749b-fe1a-4132-81ec-042cad024f81-crio-socket\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-image-registry-private-configuration\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292330    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9007b6e3-863d-426c-b9c7-c2c3637861af-config-volume\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292376    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-registry-tls\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292440    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-hub\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/46efc957-0cdf-4401-a831-0820448ca283-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292494    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-registry-certificates\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292519    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4m9f\" (UniqueName: \"kubernetes.io/projected/5c025bbe-ee0c-428a-841f-98c633069cbb-kube-api-access-z4m9f\") pod \"managed-serviceaccount-addon-agent-68f54f5f8-4s6wn\" (UID: \"5c025bbe-ee0c-428a-841f-98c633069cbb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"
Jul 30 22:41:15.292556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9007b6e3-863d-426c-b9c7-c2c3637861af-metrics-tls\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.293116 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-ca-trust-extracted\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.293116 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292594    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.293116 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292623    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2j7vv\" (UniqueName: \"kubernetes.io/projected/c82b749b-fe1a-4132-81ec-042cad024f81-kube-api-access-2j7vv\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.293116 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.292751    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c82b749b-fe1a-4132-81ec-042cad024f81-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.296038 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.296016    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c82b749b-fe1a-4132-81ec-042cad024f81-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.296159 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.296100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f931cf0f-d228-45b8-999a-171024393c99-cert\") pod \"ingress-canary-jzsrh\" (UID: \"f931cf0f-d228-45b8-999a-171024393c99\") " pod="openshift-ingress-canary/ingress-canary-jzsrh"
Jul 30 22:41:15.302415 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.302395    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2j7vv\" (UniqueName: \"kubernetes.io/projected/c82b749b-fe1a-4132-81ec-042cad024f81-kube-api-access-2j7vv\") pod \"insights-runtime-extractor-rwrlr\" (UID: \"c82b749b-fe1a-4132-81ec-042cad024f81\") " pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.302627 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.302609    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szn49\" (UniqueName: \"kubernetes.io/projected/f931cf0f-d228-45b8-999a-171024393c99-kube-api-access-szn49\") pod \"ingress-canary-jzsrh\" (UID: \"f931cf0f-d228-45b8-999a-171024393c99\") " pod="openshift-ingress-canary/ingress-canary-jzsrh"
Jul 30 22:41:15.327910 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.327876    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jzsrh"
Jul 30 22:41:15.369098 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.369063    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rwrlr"
Jul 30 22:41:15.393469 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s795t\" (UniqueName: \"kubernetes.io/projected/46efc957-0cdf-4401-a831-0820448ca283-kube-api-access-s795t\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.393622 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393482    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-ca\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.393622 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9007b6e3-863d-426c-b9c7-c2c3637861af-tmp-dir\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.393622 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.393622 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-bound-sa-token\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.393854 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-trusted-ca\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.393915 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9007b6e3-863d-426c-b9c7-c2c3637861af-tmp-dir\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.393915 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393906    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-image-registry-private-configuration\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.394025 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9007b6e3-863d-426c-b9c7-c2c3637861af-config-volume\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.394025 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.393979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-registry-tls\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.394148 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-hub\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.394148 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/46efc957-0cdf-4401-a831-0820448ca283-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.394148 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-registry-certificates\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.394148 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394134    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4m9f\" (UniqueName: \"kubernetes.io/projected/5c025bbe-ee0c-428a-841f-98c633069cbb-kube-api-access-z4m9f\") pod \"managed-serviceaccount-addon-agent-68f54f5f8-4s6wn\" (UID: \"5c025bbe-ee0c-428a-841f-98c633069cbb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"
Jul 30 22:41:15.394333 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394160    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9007b6e3-863d-426c-b9c7-c2c3637861af-metrics-tls\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.394333 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-ca-trust-extracted\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.394333 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.394333 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394248    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7n2q\" (UniqueName: \"kubernetes.io/projected/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-kube-api-access-c7n2q\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.394333 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394277    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c025bbe-ee0c-428a-841f-98c633069cbb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68f54f5f8-4s6wn\" (UID: \"5c025bbe-ee0c-428a-841f-98c633069cbb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"
Jul 30 22:41:15.394333 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394307    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5k9b\" (UniqueName: \"kubernetes.io/projected/9007b6e3-863d-426c-b9c7-c2c3637861af-kube-api-access-g5k9b\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.396807 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.394509    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-installation-pull-secrets\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.396807 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.395018    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-trusted-ca\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.396807 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.396742    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-ca-trust-extracted\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.397422 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.397394    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/46efc957-0cdf-4401-a831-0820448ca283-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.397546 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.397525    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-registry-certificates\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.397693 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.397674    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-ca\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.397811 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.397744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9007b6e3-863d-426c-b9c7-c2c3637861af-config-volume\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.402082 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.402015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-registry-tls\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.402330 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.402307    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.403687 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.403350    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5c025bbe-ee0c-428a-841f-98c633069cbb-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68f54f5f8-4s6wn\" (UID: \"5c025bbe-ee0c-428a-841f-98c633069cbb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"
Jul 30 22:41:15.404148 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.404098    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s795t\" (UniqueName: \"kubernetes.io/projected/46efc957-0cdf-4401-a831-0820448ca283-kube-api-access-s795t\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.406041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.405426    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.406041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.405734    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9007b6e3-863d-426c-b9c7-c2c3637861af-metrics-tls\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.406041 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.405998    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/46efc957-0cdf-4401-a831-0820448ca283-hub\") pod \"cluster-proxy-proxy-agent-7df99885d8-x2p8s\" (UID: \"46efc957-0cdf-4401-a831-0820448ca283\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.406947 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.406895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-installation-pull-secrets\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.407994 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.407943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-image-registry-private-configuration\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.407994 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.407952    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5k9b\" (UniqueName: \"kubernetes.io/projected/9007b6e3-863d-426c-b9c7-c2c3637861af-kube-api-access-g5k9b\") pod \"dns-default-tm4q5\" (UID: \"9007b6e3-863d-426c-b9c7-c2c3637861af\") " pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.408218 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.408194    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-bound-sa-token\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.409107 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.409073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4m9f\" (UniqueName: \"kubernetes.io/projected/5c025bbe-ee0c-428a-841f-98c633069cbb-kube-api-access-z4m9f\") pod \"managed-serviceaccount-addon-agent-68f54f5f8-4s6wn\" (UID: \"5c025bbe-ee0c-428a-841f-98c633069cbb\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"
Jul 30 22:41:15.409642 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.409611    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7n2q\" (UniqueName: \"kubernetes.io/projected/5cb49bc7-91fd-4326-ad9c-30dc316e9a98-kube-api-access-c7n2q\") pod \"image-registry-6758c75797-xh58q\" (UID: \"5cb49bc7-91fd-4326-ad9c-30dc316e9a98\") " pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.442825 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.442343    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:15.453891 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.453251    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"
Jul 30 22:41:15.459871 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.459618    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"
Jul 30 22:41:15.479810 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.479007    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:15.493655 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.493574    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jzsrh"]
Jul 30 22:41:15.503655 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:15.503618    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf931cf0f_d228_45b8_999a_171024393c99.slice/crio-dd9faa66c873730a3ad6d2581f2292810d2c23e5b60d114a9c2c6faf8ab3897c WatchSource:0}: Error finding container dd9faa66c873730a3ad6d2581f2292810d2c23e5b60d114a9c2c6faf8ab3897c: Status 404 returned error can't find the container with id dd9faa66c873730a3ad6d2581f2292810d2c23e5b60d114a9c2c6faf8ab3897c
Jul 30 22:41:15.571823 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.571607    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rwrlr"]
Jul 30 22:41:15.575437 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.575361    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jzsrh" event={"ID":"f931cf0f-d228-45b8-999a-171024393c99","Type":"ContainerStarted","Data":"dd9faa66c873730a3ad6d2581f2292810d2c23e5b60d114a9c2c6faf8ab3897c"}
Jul 30 22:41:15.580161 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:15.580135    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82b749b_fe1a_4132_81ec_042cad024f81.slice/crio-4cc22a7c3bab16b327ebabbdf2be559bb3e18ba255d45a22886f25510af3fd4e WatchSource:0}: Error finding container 4cc22a7c3bab16b327ebabbdf2be559bb3e18ba255d45a22886f25510af3fd4e: Status 404 returned error can't find the container with id 4cc22a7c3bab16b327ebabbdf2be559bb3e18ba255d45a22886f25510af3fd4e
Jul 30 22:41:15.623179 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.623144    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tm4q5"]
Jul 30 22:41:15.628658 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:15.628631    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9007b6e3_863d_426c_b9c7_c2c3637861af.slice/crio-c7e8220fdaca7cd6b192102c094518550ef83c317da759bc450d0ab4a96d906c WatchSource:0}: Error finding container c7e8220fdaca7cd6b192102c094518550ef83c317da759bc450d0ab4a96d906c: Status 404 returned error can't find the container with id c7e8220fdaca7cd6b192102c094518550ef83c317da759bc450d0ab4a96d906c
Jul 30 22:41:15.677278 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.677224    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6758c75797-xh58q"]
Jul 30 22:41:15.681356 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:15.681334    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb49bc7_91fd_4326_ad9c_30dc316e9a98.slice/crio-b80c6626277f3313d611a615137f23a40c6ea64b1ef3022054a73d725c8ea2c3 WatchSource:0}: Error finding container b80c6626277f3313d611a615137f23a40c6ea64b1ef3022054a73d725c8ea2c3: Status 404 returned error can't find the container with id b80c6626277f3313d611a615137f23a40c6ea64b1ef3022054a73d725c8ea2c3
Jul 30 22:41:15.847295 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.847215    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn"]
Jul 30 22:41:15.850332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:15.849703    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s"]
Jul 30 22:41:15.851735 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:15.851707    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c025bbe_ee0c_428a_841f_98c633069cbb.slice/crio-25c7f5335f60fd792c9729843197d54f088c71944c316b71a83bff6b33e207a2 WatchSource:0}: Error finding container 25c7f5335f60fd792c9729843197d54f088c71944c316b71a83bff6b33e207a2: Status 404 returned error can't find the container with id 25c7f5335f60fd792c9729843197d54f088c71944c316b71a83bff6b33e207a2
Jul 30 22:41:15.852809 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:15.852785    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46efc957_0cdf_4401_a831_0820448ca283.slice/crio-e17643a6076a40573dffff8ea629ed1402e6b391ec032add87a5391e1be9d6d2 WatchSource:0}: Error finding container e17643a6076a40573dffff8ea629ed1402e6b391ec032add87a5391e1be9d6d2: Status 404 returned error can't find the container with id e17643a6076a40573dffff8ea629ed1402e6b391ec032add87a5391e1be9d6d2
Jul 30 22:41:16.581066 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.580962    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tm4q5" event={"ID":"9007b6e3-863d-426c-b9c7-c2c3637861af","Type":"ContainerStarted","Data":"c7e8220fdaca7cd6b192102c094518550ef83c317da759bc450d0ab4a96d906c"}
Jul 30 22:41:16.582608 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.582548    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn" event={"ID":"5c025bbe-ee0c-428a-841f-98c633069cbb","Type":"ContainerStarted","Data":"25c7f5335f60fd792c9729843197d54f088c71944c316b71a83bff6b33e207a2"}
Jul 30 22:41:16.584493 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.584463    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6758c75797-xh58q" event={"ID":"5cb49bc7-91fd-4326-ad9c-30dc316e9a98","Type":"ContainerStarted","Data":"e6540fd438a94a86216a6d31b6c3f24a801dd8831d92f236294c66bf114bdf94"}
Jul 30 22:41:16.584600 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.584515    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6758c75797-xh58q" event={"ID":"5cb49bc7-91fd-4326-ad9c-30dc316e9a98","Type":"ContainerStarted","Data":"b80c6626277f3313d611a615137f23a40c6ea64b1ef3022054a73d725c8ea2c3"}
Jul 30 22:41:16.584667 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.584635    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:41:16.587254 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.587228    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s" event={"ID":"46efc957-0cdf-4401-a831-0820448ca283","Type":"ContainerStarted","Data":"e17643a6076a40573dffff8ea629ed1402e6b391ec032add87a5391e1be9d6d2"}
Jul 30 22:41:16.590253 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.590228    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rwrlr" event={"ID":"c82b749b-fe1a-4132-81ec-042cad024f81","Type":"ContainerStarted","Data":"58ea3e5c5f3a81d7c57db9536decf60a25384ce86be90e101314038396e49ea4"}
Jul 30 22:41:16.590363 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.590285    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rwrlr" event={"ID":"c82b749b-fe1a-4132-81ec-042cad024f81","Type":"ContainerStarted","Data":"128e7a24451529f71020ce3ac3c35b8d10ed0972e6643b0b7b2f67a014e32ee4"}
Jul 30 22:41:16.590363 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.590315    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rwrlr" event={"ID":"c82b749b-fe1a-4132-81ec-042cad024f81","Type":"ContainerStarted","Data":"4cc22a7c3bab16b327ebabbdf2be559bb3e18ba255d45a22886f25510af3fd4e"}
Jul 30 22:41:16.605644 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:16.605584    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6758c75797-xh58q" podStartSLOduration=1.605563766 podStartE2EDuration="1.605563766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:41:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:16.603729441 +0000 UTC m=+56.778693121" watchObservedRunningTime="2026-07-30 22:41:16.605563766 +0000 UTC m=+56.780527454"
Jul 30 22:41:20.520086 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:20.520030    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xj7kf"
Jul 30 22:41:22.609912 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.609822    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tm4q5" event={"ID":"9007b6e3-863d-426c-b9c7-c2c3637861af","Type":"ContainerStarted","Data":"7c4a9896017e292064567fc070c6cddf428e12f2cae992748e9dacdd4bf882fa"}
Jul 30 22:41:22.609912 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.609909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tm4q5" event={"ID":"9007b6e3-863d-426c-b9c7-c2c3637861af","Type":"ContainerStarted","Data":"b2ec6d1b850075ac3eae6e98fbfd7ef91209ebdf1af432da58e24fcc763190e3"}
Jul 30 22:41:22.610424 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.609938    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:22.611348 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.611321    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jzsrh" event={"ID":"f931cf0f-d228-45b8-999a-171024393c99","Type":"ContainerStarted","Data":"ac9a72176918ce990db8955b29c84d7fe15fe48aeecb58e4c34f2f5a8787ce4b"}
Jul 30 22:41:22.612714 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.612683    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn" event={"ID":"5c025bbe-ee0c-428a-841f-98c633069cbb","Type":"ContainerStarted","Data":"7c727fbdf1bc6dd1121fc0980c521986304e6f5fe90f8390660ee339f3582341"}
Jul 30 22:41:22.614535 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.614518    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s" event={"ID":"46efc957-0cdf-4401-a831-0820448ca283","Type":"ContainerStarted","Data":"ec0638f8b19a16242266312a50da661a7b9627380896d662a48cd3229ade031a"}
Jul 30 22:41:22.614609 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.614546    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s" event={"ID":"46efc957-0cdf-4401-a831-0820448ca283","Type":"ContainerStarted","Data":"49e2390aaa722d480412feb6b3bccbeb76b75cf9ff9c1b0373e2eda05e826351"}
Jul 30 22:41:22.614609 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.614557    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s" event={"ID":"46efc957-0cdf-4401-a831-0820448ca283","Type":"ContainerStarted","Data":"7d1603ba2e94a1421fe3a4dfe39a4fd1b77625e1b8ee99e176a34ff8ae46a60b"}
Jul 30 22:41:22.616367 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.616348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rwrlr" event={"ID":"c82b749b-fe1a-4132-81ec-042cad024f81","Type":"ContainerStarted","Data":"a69fc2022419312e1866d2b750f73ee06d0a5c62db94d95cc9ac1b3c9d0982cf"}
Jul 30 22:41:22.634792 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.634647    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tm4q5" podStartSLOduration=1.15643023 podStartE2EDuration="7.634631164s" totalImagesPullingTime="6.478200934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:22.634198759 +0000 UTC m=+62.809162450" watchObservedRunningTime="2026-07-30 22:41:22.634631164 +0000 UTC m=+62.809594820"
Jul 30 22:41:22.656858 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.656801    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rwrlr" podStartSLOduration=1.317740253 podStartE2EDuration="7.656785234s" totalImagesPullingTime="6.339044981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:41:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:22.656412903 +0000 UTC m=+62.831376582" watchObservedRunningTime="2026-07-30 22:41:22.656785234 +0000 UTC m=+62.831748912"
Jul 30 22:41:22.698702 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.698646    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7df99885d8-x2p8s" podStartSLOduration=1.421405166 podStartE2EDuration="7.6986207s" totalImagesPullingTime="6.277215534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:22.698177575 +0000 UTC m=+62.873141256" watchObservedRunningTime="2026-07-30 22:41:22.6986207 +0000 UTC m=+62.873584380"
Jul 30 22:41:22.699882 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.699838    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jzsrh" podStartSLOduration=2.095855468 podStartE2EDuration="8.699823839s" totalImagesPullingTime="6.603968371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:41:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:22.677771848 +0000 UTC m=+62.852735525" watchObservedRunningTime="2026-07-30 22:41:22.699823839 +0000 UTC m=+62.874787520"
Jul 30 22:41:22.721366 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:22.721317    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68f54f5f8-4s6wn" podStartSLOduration=1.4522279089999999 podStartE2EDuration="7.721300644s" totalImagesPullingTime="6.269072735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:22.720516145 +0000 UTC m=+62.895479823" watchObservedRunningTime="2026-07-30 22:41:22.721300644 +0000 UTC m=+62.896264323"
Jul 30 22:41:26.095108 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.095060    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:41:26.095108 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.095113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:41:26.095641 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.095156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:41:26.098804 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.098783    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 30 22:41:26.099023 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.099010    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 30 22:41:26.099265 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.099249    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 30 22:41:26.108643 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.108620    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b062ccea-8395-4cf7-8ad8-6fe3011f432b-metrics-certs\") pod \"network-metrics-daemon-m7mw5\" (UID: \"b062ccea-8395-4cf7-8ad8-6fe3011f432b\") " pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:41:26.108743 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.108698    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/33ceed57-7068-43fe-82bd-ed7a69ce892d-original-pull-secret\") pod \"global-pull-secret-syncer-989b2\" (UID: \"33ceed57-7068-43fe-82bd-ed7a69ce892d\") " pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:41:26.108857 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.108843    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 30 22:41:26.119447 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.119427    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2z2t\" (UniqueName: \"kubernetes.io/projected/63ad352e-50fa-49c7-9dfa-dcff5633ab8e-kube-api-access-f2z2t\") pod \"network-check-target-z2wbw\" (UID: \"63ad352e-50fa-49c7-9dfa-dcff5633ab8e\") " pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:41:26.294192 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.294157    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-29hrf\""
Jul 30 22:41:26.301534 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.301504    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m7mw5"
Jul 30 22:41:26.301534 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.301524    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-989b2"
Jul 30 22:41:26.315778 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.315749    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzp2k\""
Jul 30 22:41:26.323929 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.323906    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:41:26.477834 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.477758    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-989b2"]
Jul 30 22:41:26.482268 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:26.482218    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ceed57_7068_43fe_82bd_ed7a69ce892d.slice/crio-0a33ce140542b6de7389059658c81751d608b246355c0db6b4e3c35e04e80b3c WatchSource:0}: Error finding container 0a33ce140542b6de7389059658c81751d608b246355c0db6b4e3c35e04e80b3c: Status 404 returned error can't find the container with id 0a33ce140542b6de7389059658c81751d608b246355c0db6b4e3c35e04e80b3c
Jul 30 22:41:26.515524 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.515494    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m7mw5"]
Jul 30 22:41:26.518904 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:26.518877    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb062ccea_8395_4cf7_8ad8_6fe3011f432b.slice/crio-d128d26a43cbf9efcf95f23777642eaa96724dd48450943ef289a0fbfbc3008a WatchSource:0}: Error finding container d128d26a43cbf9efcf95f23777642eaa96724dd48450943ef289a0fbfbc3008a: Status 404 returned error can't find the container with id d128d26a43cbf9efcf95f23777642eaa96724dd48450943ef289a0fbfbc3008a
Jul 30 22:41:26.547330 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.547298    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z2wbw"]
Jul 30 22:41:26.552231 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:26.552208    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ad352e_50fa_49c7_9dfa_dcff5633ab8e.slice/crio-e41f3e39b1150315f8762ff4ec31045eb27df897d255973baf1ebccd0833a727 WatchSource:0}: Error finding container e41f3e39b1150315f8762ff4ec31045eb27df897d255973baf1ebccd0833a727: Status 404 returned error can't find the container with id e41f3e39b1150315f8762ff4ec31045eb27df897d255973baf1ebccd0833a727
Jul 30 22:41:26.632322 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.632226    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m7mw5" event={"ID":"b062ccea-8395-4cf7-8ad8-6fe3011f432b","Type":"ContainerStarted","Data":"d128d26a43cbf9efcf95f23777642eaa96724dd48450943ef289a0fbfbc3008a"}
Jul 30 22:41:26.633314 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.633286    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-989b2" event={"ID":"33ceed57-7068-43fe-82bd-ed7a69ce892d","Type":"ContainerStarted","Data":"0a33ce140542b6de7389059658c81751d608b246355c0db6b4e3c35e04e80b3c"}
Jul 30 22:41:26.634396 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:26.634376    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z2wbw" event={"ID":"63ad352e-50fa-49c7-9dfa-dcff5633ab8e","Type":"ContainerStarted","Data":"e41f3e39b1150315f8762ff4ec31045eb27df897d255973baf1ebccd0833a727"}
Jul 30 22:41:28.644977 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:28.644940    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m7mw5" event={"ID":"b062ccea-8395-4cf7-8ad8-6fe3011f432b","Type":"ContainerStarted","Data":"2e5636da1d0d1a3a568acdad2d489808eccc0088ec045de815f425523055f768"}
Jul 30 22:41:28.645490 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:28.645003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m7mw5" event={"ID":"b062ccea-8395-4cf7-8ad8-6fe3011f432b","Type":"ContainerStarted","Data":"3f351fc7219f4c056e312a4fda17d6dfd35d84ba37e5bee7af1c21d6a4b8ba7c"}
Jul 30 22:41:28.663299 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:28.663235    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-m7mw5" podStartSLOduration=67.514661972 podStartE2EDuration="1m8.663214399s" totalImagesPullingTime="1.148552427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:28.662363379 +0000 UTC m=+68.837327060" watchObservedRunningTime="2026-07-30 22:41:28.663214399 +0000 UTC m=+68.838178079"
Jul 30 22:41:31.657871 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:31.657766    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z2wbw" event={"ID":"63ad352e-50fa-49c7-9dfa-dcff5633ab8e","Type":"ContainerStarted","Data":"ccdd23829d12a6eae4dfe9eee7f6e389198ebb21a4a3848b5d3ec002e3c8908e"}
Jul 30 22:41:31.657871 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:31.657834    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:41:31.659181 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:31.659159    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-989b2" event={"ID":"33ceed57-7068-43fe-82bd-ed7a69ce892d","Type":"ContainerStarted","Data":"e31eaf6ab8affab39f7b615fabf344d007b3d3d81a443b946cba6432eed31349"}
Jul 30 22:41:31.692838 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:31.692770    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-989b2" podStartSLOduration=66.975440048 podStartE2EDuration="1m11.692746996s" totalImagesPullingTime="4.717306948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:31.692334245 +0000 UTC m=+71.867297936" watchObservedRunningTime="2026-07-30 22:41:31.692746996 +0000 UTC m=+71.867710678"
Jul 30 22:41:31.693435 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:31.693389    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z2wbw" podStartSLOduration=67.048840769 podStartE2EDuration="1m11.693378141s" totalImagesPullingTime="4.644537372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:31.673511894 +0000 UTC m=+71.848475573" watchObservedRunningTime="2026-07-30 22:41:31.693378141 +0000 UTC m=+71.868341821"
Jul 30 22:41:32.621137 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:32.621108    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tm4q5"
Jul 30 22:41:33.833965 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.833931    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-48hk5"]
Jul 30 22:41:33.836090 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.836071    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:33.843985 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.843963    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 30 22:41:33.844180 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.844165    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 30 22:41:33.845099 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.845081    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 30 22:41:33.845324 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.845309    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 30 22:41:33.845491 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.845466    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 30 22:41:33.845550 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.845533    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 30 22:41:33.845820 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.845806    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-94cf2\""
Jul 30 22:41:33.953117 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.953076    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psb7g\" (UniqueName: \"kubernetes.io/projected/115e7dcd-3593-4794-8ebe-0022a1494f91-kube-api-access-psb7g\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:33.953117 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.953118    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-tls\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:33.953332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.953137    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-wtmp\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:33.953332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.953158    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/115e7dcd-3593-4794-8ebe-0022a1494f91-root\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:33.953332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.953222    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/115e7dcd-3593-4794-8ebe-0022a1494f91-sys\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:33.953332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.953269    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-textfile\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:33.953332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.953293    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/115e7dcd-3593-4794-8ebe-0022a1494f91-metrics-client-ca\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:33.953332 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.953315    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:33.953529 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:33.953339    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-accelerators-collector-config\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.054161 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.054116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-accelerators-collector-config\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.054360 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.054193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psb7g\" (UniqueName: \"kubernetes.io/projected/115e7dcd-3593-4794-8ebe-0022a1494f91-kube-api-access-psb7g\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.054360 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.054246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-tls\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.054360 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.054276    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-wtmp\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.054360 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.054315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/115e7dcd-3593-4794-8ebe-0022a1494f91-root\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.054360 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.054356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/115e7dcd-3593-4794-8ebe-0022a1494f91-sys\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.054614 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.054409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-textfile\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.054614 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.054439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/115e7dcd-3593-4794-8ebe-0022a1494f91-metrics-client-ca\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.054614 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.054473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.057844 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.057816    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-wtmp\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.058086 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.057883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/115e7dcd-3593-4794-8ebe-0022a1494f91-root\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.058424 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.057917    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-textfile\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.058504 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.057924    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/115e7dcd-3593-4794-8ebe-0022a1494f91-sys\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.058586 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.058565    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.058678 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.058654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/115e7dcd-3593-4794-8ebe-0022a1494f91-metrics-client-ca\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.058750 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.058655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-accelerators-collector-config\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.060641 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.060623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/115e7dcd-3593-4794-8ebe-0022a1494f91-node-exporter-tls\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.068145 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.068118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psb7g\" (UniqueName: \"kubernetes.io/projected/115e7dcd-3593-4794-8ebe-0022a1494f91-kube-api-access-psb7g\") pod \"node-exporter-48hk5\" (UID: \"115e7dcd-3593-4794-8ebe-0022a1494f91\") " pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.146000 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.145904    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-48hk5"
Jul 30 22:41:34.156072 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:41:34.156020    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115e7dcd_3593_4794_8ebe_0022a1494f91.slice/crio-2f864fbc2c7c4e7bac7f6c3fbc9bd4e00d68e6cd7c0f0936741e6a6e1850b804 WatchSource:0}: Error finding container 2f864fbc2c7c4e7bac7f6c3fbc9bd4e00d68e6cd7c0f0936741e6a6e1850b804: Status 404 returned error can't find the container with id 2f864fbc2c7c4e7bac7f6c3fbc9bd4e00d68e6cd7c0f0936741e6a6e1850b804
Jul 30 22:41:34.670398 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:34.670363    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-48hk5" event={"ID":"115e7dcd-3593-4794-8ebe-0022a1494f91","Type":"ContainerStarted","Data":"2f864fbc2c7c4e7bac7f6c3fbc9bd4e00d68e6cd7c0f0936741e6a6e1850b804"}
Jul 30 22:41:35.484935 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:35.484897    2578 patch_prober.go:28] interesting pod/image-registry-6758c75797-xh58q container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 30 22:41:35.485338 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:35.484951    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6758c75797-xh58q" podUID="5cb49bc7-91fd-4326-ad9c-30dc316e9a98" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 22:41:35.675720 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:35.675633    2578 generic.go:356] "Generic (PLEG): container finished" podID="115e7dcd-3593-4794-8ebe-0022a1494f91" containerID="f6667008febb4aa572efac446f3827cb7f016ad964730246f08475fceba0cacd" exitCode=0
Jul 30 22:41:35.675875 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:35.675724    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-48hk5" event={"ID":"115e7dcd-3593-4794-8ebe-0022a1494f91","Type":"ContainerDied","Data":"f6667008febb4aa572efac446f3827cb7f016ad964730246f08475fceba0cacd"}
Jul 30 22:41:36.680950 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:36.680918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-48hk5" event={"ID":"115e7dcd-3593-4794-8ebe-0022a1494f91","Type":"ContainerStarted","Data":"c5a4ec3e3b9918bb93495be82e041264fb4ace022055af2cb8220983d30f9bd8"}
Jul 30 22:41:36.681366 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:36.680968    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-48hk5" event={"ID":"115e7dcd-3593-4794-8ebe-0022a1494f91","Type":"ContainerStarted","Data":"b8e00ab3f01cc522ec4603beed3fccb67ca25aba685095e1005a1b0119d45a9a"}
Jul 30 22:41:36.708318 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:36.708260    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-48hk5" podStartSLOduration=2.538953429 podStartE2EDuration="3.708238875s" totalImagesPullingTime="1.097384239s" totalInitContainerRuntime="71.901207ms" isStatefulPod=true podCreationTimestamp="2026-07-30 22:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:41:36.707807623 +0000 UTC m=+76.882771298" watchObservedRunningTime="2026-07-30 22:41:36.708238875 +0000 UTC m=+76.883202596"
Jul 30 22:41:37.599170 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:41:37.599144    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6758c75797-xh58q"
Jul 30 22:42:02.665471 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:42:02.665433    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z2wbw"
Jul 30 22:45:20.196895 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:20.196861    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:45:20.199433 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:20.199408    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:45:20.202345 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:20.202327    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 30 22:45:37.678317 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.678284    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"]
Jul 30 22:45:37.680032 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.680016    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"
Jul 30 22:45:37.682543 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.682519    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 30 22:45:37.682721 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.682703    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 30 22:45:37.683473 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.683453    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-d7fn6\""
Jul 30 22:45:37.690933 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.690898    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"]
Jul 30 22:45:37.814237 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.814194    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e41dba6a-cad0-40de-af0c-e13a2ca8c041-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6lntc\" (UID: \"e41dba6a-cad0-40de-af0c-e13a2ca8c041\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"
Jul 30 22:45:37.814409 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.814256    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8dnz\" (UniqueName: \"kubernetes.io/projected/e41dba6a-cad0-40de-af0c-e13a2ca8c041-kube-api-access-p8dnz\") pod \"openshift-lws-operator-fd8ccff4c-6lntc\" (UID: \"e41dba6a-cad0-40de-af0c-e13a2ca8c041\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"
Jul 30 22:45:37.915472 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.915427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8dnz\" (UniqueName: \"kubernetes.io/projected/e41dba6a-cad0-40de-af0c-e13a2ca8c041-kube-api-access-p8dnz\") pod \"openshift-lws-operator-fd8ccff4c-6lntc\" (UID: \"e41dba6a-cad0-40de-af0c-e13a2ca8c041\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"
Jul 30 22:45:37.915606 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.915500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e41dba6a-cad0-40de-af0c-e13a2ca8c041-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6lntc\" (UID: \"e41dba6a-cad0-40de-af0c-e13a2ca8c041\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"
Jul 30 22:45:37.915831 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.915816    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e41dba6a-cad0-40de-af0c-e13a2ca8c041-tmp\") pod \"openshift-lws-operator-fd8ccff4c-6lntc\" (UID: \"e41dba6a-cad0-40de-af0c-e13a2ca8c041\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"
Jul 30 22:45:37.923981 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.923948    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8dnz\" (UniqueName: \"kubernetes.io/projected/e41dba6a-cad0-40de-af0c-e13a2ca8c041-kube-api-access-p8dnz\") pod \"openshift-lws-operator-fd8ccff4c-6lntc\" (UID: \"e41dba6a-cad0-40de-af0c-e13a2ca8c041\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"
Jul 30 22:45:37.994001 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:37.993958    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"
Jul 30 22:45:38.121243 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:38.121209    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc"]
Jul 30 22:45:38.124819 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:45:38.124789    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode41dba6a_cad0_40de_af0c_e13a2ca8c041.slice/crio-ca93c054425089d7208ec0f8c62fbe505c4da88a0ec32079ab21cd2402b58e05 WatchSource:0}: Error finding container ca93c054425089d7208ec0f8c62fbe505c4da88a0ec32079ab21cd2402b58e05: Status 404 returned error can't find the container with id ca93c054425089d7208ec0f8c62fbe505c4da88a0ec32079ab21cd2402b58e05
Jul 30 22:45:38.126418 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:38.126399    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 22:45:38.451904 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:38.451860    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc" event={"ID":"e41dba6a-cad0-40de-af0c-e13a2ca8c041","Type":"ContainerStarted","Data":"ca93c054425089d7208ec0f8c62fbe505c4da88a0ec32079ab21cd2402b58e05"}
Jul 30 22:45:41.466087 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:41.466032    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc" event={"ID":"e41dba6a-cad0-40de-af0c-e13a2ca8c041","Type":"ContainerStarted","Data":"c694fff1d8f882f9c70fae04a463f99e12e3c601abda25f992729ab9660f704c"}
Jul 30 22:45:41.482163 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:41.482114    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-6lntc" podStartSLOduration=1.9442699989999999 podStartE2EDuration="4.482097886s" totalImagesPullingTime="2.537827887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:45:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:45:41.48039571 +0000 UTC m=+321.655359388" watchObservedRunningTime="2026-07-30 22:45:41.482097886 +0000 UTC m=+321.657061565"
Jul 30 22:45:59.942717 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:59.942681    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"]
Jul 30 22:45:59.947029 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:59.947007    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:45:59.950494 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:59.950473    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 30 22:45:59.951498 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:59.951475    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 30 22:45:59.951626 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:59.951537    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 30 22:45:59.951626 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:59.951604    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jzx8w\""
Jul 30 22:45:59.952919 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:59.952902    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 30 22:45:59.962992 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:45:59.962964    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"]
Jul 30 22:46:00.075096 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.075034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b671cb7b-cf47-4a48-b297-bd65c9b46755-webhook-cert\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.075096 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.075101    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b671cb7b-cf47-4a48-b297-bd65c9b46755-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.075345 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.075217    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfv6h\" (UniqueName: \"kubernetes.io/projected/b671cb7b-cf47-4a48-b297-bd65c9b46755-kube-api-access-cfv6h\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.075345 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.075258    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b671cb7b-cf47-4a48-b297-bd65c9b46755-tmp\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.075345 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.075307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b671cb7b-cf47-4a48-b297-bd65c9b46755-manifests\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.176516 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.176474    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b671cb7b-cf47-4a48-b297-bd65c9b46755-manifests\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.176714 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.176539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b671cb7b-cf47-4a48-b297-bd65c9b46755-webhook-cert\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.176714 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.176572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b671cb7b-cf47-4a48-b297-bd65c9b46755-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.176714 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.176618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfv6h\" (UniqueName: \"kubernetes.io/projected/b671cb7b-cf47-4a48-b297-bd65c9b46755-kube-api-access-cfv6h\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.176714 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.176642    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b671cb7b-cf47-4a48-b297-bd65c9b46755-tmp\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.176936 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.176860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b671cb7b-cf47-4a48-b297-bd65c9b46755-manifests\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.179084 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.179032    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b671cb7b-cf47-4a48-b297-bd65c9b46755-tmp\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.179279 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.179261    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b671cb7b-cf47-4a48-b297-bd65c9b46755-apiservice-cert\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.179348 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.179327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b671cb7b-cf47-4a48-b297-bd65c9b46755-webhook-cert\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.187066 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.187024    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfv6h\" (UniqueName: \"kubernetes.io/projected/b671cb7b-cf47-4a48-b297-bd65c9b46755-kube-api-access-cfv6h\") pod \"opendatahub-operator-controller-manager-ff8c5f475-ghbrv\" (UID: \"b671cb7b-cf47-4a48-b297-bd65c9b46755\") " pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.258166 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.258043    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:00.404607 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.404571    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"]
Jul 30 22:46:00.407640 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:46:00.407611    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb671cb7b_cf47_4a48_b297_bd65c9b46755.slice/crio-ae40936228aa6b18aa634947993a4efc7c67d8150adc2d1680d15926ff13dcb8 WatchSource:0}: Error finding container ae40936228aa6b18aa634947993a4efc7c67d8150adc2d1680d15926ff13dcb8: Status 404 returned error can't find the container with id ae40936228aa6b18aa634947993a4efc7c67d8150adc2d1680d15926ff13dcb8
Jul 30 22:46:00.529023 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:00.528941    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv" event={"ID":"b671cb7b-cf47-4a48-b297-bd65c9b46755","Type":"ContainerStarted","Data":"ae40936228aa6b18aa634947993a4efc7c67d8150adc2d1680d15926ff13dcb8"}
Jul 30 22:46:03.543164 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:03.543127    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv" event={"ID":"b671cb7b-cf47-4a48-b297-bd65c9b46755","Type":"ContainerStarted","Data":"13a04387d1403788a653c1892d7cc1adde61fe0703499f7f2248d8636e64a530"}
Jul 30 22:46:05.551853 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:05.551815    2578 generic.go:356] "Generic (PLEG): container finished" podID="b671cb7b-cf47-4a48-b297-bd65c9b46755" containerID="13a04387d1403788a653c1892d7cc1adde61fe0703499f7f2248d8636e64a530" exitCode=0
Jul 30 22:46:05.552280 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:05.551871    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv" event={"ID":"b671cb7b-cf47-4a48-b297-bd65c9b46755","Type":"ContainerDied","Data":"13a04387d1403788a653c1892d7cc1adde61fe0703499f7f2248d8636e64a530"}
Jul 30 22:46:06.557284 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:06.557251    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv" event={"ID":"b671cb7b-cf47-4a48-b297-bd65c9b46755","Type":"ContainerStarted","Data":"1c4064060338d521f60ae671104a04487b8f24d59f30a5102c8e10ff5d04248a"}
Jul 30 22:46:06.557682 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:06.557547    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:06.579645 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:06.579585    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv" podStartSLOduration=3.113892781 podStartE2EDuration="7.579563125s" totalImagesPullingTime="3.44650317s" totalInitContainerRuntime="1.019167174s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:45:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:46:06.578337444 +0000 UTC m=+346.753301124" watchObservedRunningTime="2026-07-30 22:46:06.579563125 +0000 UTC m=+346.754526805"
Jul 30 22:46:17.563651 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:17.563612    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-ff8c5f475-ghbrv"
Jul 30 22:46:19.822964 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.822932    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"]
Jul 30 22:46:19.826043 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.826028    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:19.832162 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.832141    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 22:46:19.832256 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.832188    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 30 22:46:19.832509 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.832488    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 30 22:46:19.839725 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.839707    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 30 22:46:19.856969 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.856945    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"]
Jul 30 22:46:19.912827 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.912792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0ff893d-0d69-4cf4-8dd3-6be4014b0e09-tls-certs\") pod \"kube-auth-proxy-b9c58d584-x4tth\" (UID: \"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09\") " pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:19.912827 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.912830    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0ff893d-0d69-4cf4-8dd3-6be4014b0e09-tmp\") pod \"kube-auth-proxy-b9c58d584-x4tth\" (UID: \"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09\") " pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:19.913031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:19.912903    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m52f\" (UniqueName: \"kubernetes.io/projected/f0ff893d-0d69-4cf4-8dd3-6be4014b0e09-kube-api-access-7m52f\") pod \"kube-auth-proxy-b9c58d584-x4tth\" (UID: \"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09\") " pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:20.005115 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.005083    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5w7xx"]
Jul 30 22:46:20.008044 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.008028    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:20.012313 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.012288    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mwddj\""
Jul 30 22:46:20.012527 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.012509    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 30 22:46:20.013773 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.013750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b65d94e7-4b50-41ec-9c55-cfda5fc159c3-cert\") pod \"odh-model-controller-7998c67dc4-5w7xx\" (UID: \"b65d94e7-4b50-41ec-9c55-cfda5fc159c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:20.013874 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.013795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m52f\" (UniqueName: \"kubernetes.io/projected/f0ff893d-0d69-4cf4-8dd3-6be4014b0e09-kube-api-access-7m52f\") pod \"kube-auth-proxy-b9c58d584-x4tth\" (UID: \"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09\") " pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:20.013874 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.013822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0ff893d-0d69-4cf4-8dd3-6be4014b0e09-tls-certs\") pod \"kube-auth-proxy-b9c58d584-x4tth\" (UID: \"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09\") " pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:20.013874 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.013847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94rf5\" (UniqueName: \"kubernetes.io/projected/b65d94e7-4b50-41ec-9c55-cfda5fc159c3-kube-api-access-94rf5\") pod \"odh-model-controller-7998c67dc4-5w7xx\" (UID: \"b65d94e7-4b50-41ec-9c55-cfda5fc159c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:20.013874 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.013870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0ff893d-0d69-4cf4-8dd3-6be4014b0e09-tmp\") pod \"kube-auth-proxy-b9c58d584-x4tth\" (UID: \"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09\") " pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:20.016454 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.016435    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0ff893d-0d69-4cf4-8dd3-6be4014b0e09-tmp\") pod \"kube-auth-proxy-b9c58d584-x4tth\" (UID: \"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09\") " pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:20.016644 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.016628    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0ff893d-0d69-4cf4-8dd3-6be4014b0e09-tls-certs\") pod \"kube-auth-proxy-b9c58d584-x4tth\" (UID: \"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09\") " pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:20.026356 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.026333    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5w7xx"]
Jul 30 22:46:20.036073 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.036027    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m52f\" (UniqueName: \"kubernetes.io/projected/f0ff893d-0d69-4cf4-8dd3-6be4014b0e09-kube-api-access-7m52f\") pod \"kube-auth-proxy-b9c58d584-x4tth\" (UID: \"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09\") " pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:20.115034 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.114938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94rf5\" (UniqueName: \"kubernetes.io/projected/b65d94e7-4b50-41ec-9c55-cfda5fc159c3-kube-api-access-94rf5\") pod \"odh-model-controller-7998c67dc4-5w7xx\" (UID: \"b65d94e7-4b50-41ec-9c55-cfda5fc159c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:20.115034 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.115006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b65d94e7-4b50-41ec-9c55-cfda5fc159c3-cert\") pod \"odh-model-controller-7998c67dc4-5w7xx\" (UID: \"b65d94e7-4b50-41ec-9c55-cfda5fc159c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:20.117570 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.117550    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b65d94e7-4b50-41ec-9c55-cfda5fc159c3-cert\") pod \"odh-model-controller-7998c67dc4-5w7xx\" (UID: \"b65d94e7-4b50-41ec-9c55-cfda5fc159c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:20.131891 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.131851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94rf5\" (UniqueName: \"kubernetes.io/projected/b65d94e7-4b50-41ec-9c55-cfda5fc159c3-kube-api-access-94rf5\") pod \"odh-model-controller-7998c67dc4-5w7xx\" (UID: \"b65d94e7-4b50-41ec-9c55-cfda5fc159c3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:20.134697 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.134681    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"
Jul 30 22:46:20.277563 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.277532    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-b9c58d584-x4tth"]
Jul 30 22:46:20.280669 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:46:20.280648    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0ff893d_0d69_4cf4_8dd3_6be4014b0e09.slice/crio-6a238f2ed38d04261211304711ffb6b7b9753ea29fc14efcc09d2233f086ba27 WatchSource:0}: Error finding container 6a238f2ed38d04261211304711ffb6b7b9753ea29fc14efcc09d2233f086ba27: Status 404 returned error can't find the container with id 6a238f2ed38d04261211304711ffb6b7b9753ea29fc14efcc09d2233f086ba27
Jul 30 22:46:20.322785 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.322758    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-mwddj\""
Jul 30 22:46:20.329983 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.329965    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:20.532217 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.532185    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5w7xx"]
Jul 30 22:46:20.534858 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:46:20.534835    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb65d94e7_4b50_41ec_9c55_cfda5fc159c3.slice/crio-10b2e87dae32abdd61e8fd74191b34d503b2531dd23bffd848e72f41ba84248c WatchSource:0}: Error finding container 10b2e87dae32abdd61e8fd74191b34d503b2531dd23bffd848e72f41ba84248c: Status 404 returned error can't find the container with id 10b2e87dae32abdd61e8fd74191b34d503b2531dd23bffd848e72f41ba84248c
Jul 30 22:46:20.603863 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.603821    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth" event={"ID":"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09","Type":"ContainerStarted","Data":"6a238f2ed38d04261211304711ffb6b7b9753ea29fc14efcc09d2233f086ba27"}
Jul 30 22:46:20.604830 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:20.604809    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx" event={"ID":"b65d94e7-4b50-41ec-9c55-cfda5fc159c3","Type":"ContainerStarted","Data":"10b2e87dae32abdd61e8fd74191b34d503b2531dd23bffd848e72f41ba84248c"}
Jul 30 22:46:21.371474 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.371436    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"]
Jul 30 22:46:21.374812 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.374779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.377512 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.377486    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9cvtl\""
Jul 30 22:46:21.377718 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.377700    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 30 22:46:21.377872 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.377857    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 30 22:46:21.378075 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.378040    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 30 22:46:21.378233 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.378216    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 30 22:46:21.389533 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.389504    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"]
Jul 30 22:46:21.424744 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.424708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb9fb\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-kube-api-access-kb9fb\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.424925 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.424788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.424925 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.424862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.424925 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.424891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.425119 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.424943    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.425119 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.424959    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.425119 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.424995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.525607 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.525564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.525788 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.525622    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.525788 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.525658    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.525788 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.525711    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb9fb\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-kube-api-access-kb9fb\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.525788 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.525749    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.526000 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.525797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.526000 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.525821    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.526731 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.526705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.529304 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.529254    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.529415 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.529365    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.529481 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.529456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.529481 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.529471    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.534020 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.533996    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.534498 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.534456    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb9fb\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-kube-api-access-kb9fb\") pod \"istiod-openshift-gateway-7b7995d5cd-7hkzj\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.688114 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.688082    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:21.866423 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:21.866386    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"]
Jul 30 22:46:21.871259 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:46:21.871225    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd52d78_98a2_4f44_863c_8e7a6a9a8fc0.slice/crio-b727b4a4e7c4d3bad1b7731c262182a969b45ac28a30dfdf1999105c84ba0b27 WatchSource:0}: Error finding container b727b4a4e7c4d3bad1b7731c262182a969b45ac28a30dfdf1999105c84ba0b27: Status 404 returned error can't find the container with id b727b4a4e7c4d3bad1b7731c262182a969b45ac28a30dfdf1999105c84ba0b27
Jul 30 22:46:22.614790 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:22.614753    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj" event={"ID":"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0","Type":"ContainerStarted","Data":"b727b4a4e7c4d3bad1b7731c262182a969b45ac28a30dfdf1999105c84ba0b27"}
Jul 30 22:46:24.137999 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:24.137970    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 30 22:46:24.625268 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:24.625160    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx" event={"ID":"b65d94e7-4b50-41ec-9c55-cfda5fc159c3","Type":"ContainerStarted","Data":"d10fe2c250ae3cf78b72d2c416fc7d6e0eca0600e587799dc14b96fb880dc636"}
Jul 30 22:46:24.625268 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:24.625230    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:24.626953 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:24.626920    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth" event={"ID":"f0ff893d-0d69-4cf4-8dd3-6be4014b0e09","Type":"ContainerStarted","Data":"8ac1a5c75d74ac00aa15026134b4568a1c89a0e32255247ad5deb1b84b9d4b39"}
Jul 30 22:46:24.641977 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:24.641925    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx" podStartSLOduration=2.036222425 podStartE2EDuration="5.641908547s" totalImagesPullingTime="3.605686122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:46:24.639967644 +0000 UTC m=+364.814931322" watchObservedRunningTime="2026-07-30 22:46:24.641908547 +0000 UTC m=+364.816872225"
Jul 30 22:46:24.657449 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:24.657282    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-b9c58d584-x4tth" podStartSLOduration=1.804345772 podStartE2EDuration="5.657265285s" totalImagesPullingTime="3.852919513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:46:24.655137685 +0000 UTC m=+364.830101363" watchObservedRunningTime="2026-07-30 22:46:24.657265285 +0000 UTC m=+364.832228964"
Jul 30 22:46:25.401785 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:25.401748    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 30 22:46:25.402172 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:25.401826    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 30 22:46:25.631684 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:25.631643    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj" event={"ID":"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0","Type":"ContainerStarted","Data":"115dcf14cf1bd4d18ae0990792eef0006e667bf581bb8cbc7d4a8b20258c084a"}
Jul 30 22:46:25.632002 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:25.631980    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:25.653330 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:25.653220    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj" podStartSLOduration=1.1268335760000001 podStartE2EDuration="4.653201389s" totalImagesPullingTime="3.526367813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:46:25.653157274 +0000 UTC m=+365.828120965" watchObservedRunningTime="2026-07-30 22:46:25.653201389 +0000 UTC m=+365.828165069"
Jul 30 22:46:26.642273 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:26.642242    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-7hkzj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 30 22:46:26.642711 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:26.642346    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj" podUID="6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 30 22:46:29.641423 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:29.641391    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:30.564401 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.564367    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"]
Jul 30 22:46:30.567617 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.567595    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:46:30.567854 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.567829    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"]
Jul 30 22:46:30.570512 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.570494    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-6jvwz\""
Jul 30 22:46:30.570667 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.570652    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 30 22:46:30.570711 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.570671    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:30.572941 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.572923    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pmcdx\""
Jul 30 22:46:30.573096 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.572985    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 30 22:46:30.577483 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.577461    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"]
Jul 30 22:46:30.583915 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.583892    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"]
Jul 30 22:46:30.594663 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.594637    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1780e2f1-70d4-4b00-9a5f-595c19eaee9d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jjs7h\" (UID: \"1780e2f1-70d4-4b00-9a5f-595c19eaee9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:30.594766 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.594669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7s47\" (UniqueName: \"kubernetes.io/projected/1780e2f1-70d4-4b00-9a5f-595c19eaee9d-kube-api-access-m7s47\") pod \"llmisvc-controller-manager-777c6bd7d5-jjs7h\" (UID: \"1780e2f1-70d4-4b00-9a5f-595c19eaee9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:30.594766 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.594694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7130dbaf-62b7-4d36-bbfb-a031aaa8f782-cert\") pod \"kserve-controller-manager-d6f4b65b8-wkz9w\" (UID: \"7130dbaf-62b7-4d36-bbfb-a031aaa8f782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:46:30.594845 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.594779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg5pc\" (UniqueName: \"kubernetes.io/projected/7130dbaf-62b7-4d36-bbfb-a031aaa8f782-kube-api-access-vg5pc\") pod \"kserve-controller-manager-d6f4b65b8-wkz9w\" (UID: \"7130dbaf-62b7-4d36-bbfb-a031aaa8f782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:46:30.696027 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.695983    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1780e2f1-70d4-4b00-9a5f-595c19eaee9d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jjs7h\" (UID: \"1780e2f1-70d4-4b00-9a5f-595c19eaee9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:30.696027 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.696029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7s47\" (UniqueName: \"kubernetes.io/projected/1780e2f1-70d4-4b00-9a5f-595c19eaee9d-kube-api-access-m7s47\") pod \"llmisvc-controller-manager-777c6bd7d5-jjs7h\" (UID: \"1780e2f1-70d4-4b00-9a5f-595c19eaee9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:30.696571 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.696088    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7130dbaf-62b7-4d36-bbfb-a031aaa8f782-cert\") pod \"kserve-controller-manager-d6f4b65b8-wkz9w\" (UID: \"7130dbaf-62b7-4d36-bbfb-a031aaa8f782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:46:30.696571 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.696123    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg5pc\" (UniqueName: \"kubernetes.io/projected/7130dbaf-62b7-4d36-bbfb-a031aaa8f782-kube-api-access-vg5pc\") pod \"kserve-controller-manager-d6f4b65b8-wkz9w\" (UID: \"7130dbaf-62b7-4d36-bbfb-a031aaa8f782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:46:30.696571 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:46:30.696144    2578 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 30 22:46:30.696571 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:46:30.696214    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1780e2f1-70d4-4b00-9a5f-595c19eaee9d-cert podName:1780e2f1-70d4-4b00-9a5f-595c19eaee9d nodeName:}" failed. No retries permitted until 2026-07-30 22:46:31.19619565 +0000 UTC m=+371.371159308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1780e2f1-70d4-4b00-9a5f-595c19eaee9d-cert") pod "llmisvc-controller-manager-777c6bd7d5-jjs7h" (UID: "1780e2f1-70d4-4b00-9a5f-595c19eaee9d") : secret "llmisvc-webhook-server-cert" not found
Jul 30 22:46:30.698701 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.698682    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7130dbaf-62b7-4d36-bbfb-a031aaa8f782-cert\") pod \"kserve-controller-manager-d6f4b65b8-wkz9w\" (UID: \"7130dbaf-62b7-4d36-bbfb-a031aaa8f782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:46:30.712359 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.712319    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg5pc\" (UniqueName: \"kubernetes.io/projected/7130dbaf-62b7-4d36-bbfb-a031aaa8f782-kube-api-access-vg5pc\") pod \"kserve-controller-manager-d6f4b65b8-wkz9w\" (UID: \"7130dbaf-62b7-4d36-bbfb-a031aaa8f782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:46:30.712485 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.712361    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7s47\" (UniqueName: \"kubernetes.io/projected/1780e2f1-70d4-4b00-9a5f-595c19eaee9d-kube-api-access-m7s47\") pod \"llmisvc-controller-manager-777c6bd7d5-jjs7h\" (UID: \"1780e2f1-70d4-4b00-9a5f-595c19eaee9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:30.879424 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:30.879335    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:46:31.029204 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:31.029170    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"]
Jul 30 22:46:31.032585 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:46:31.032555    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7130dbaf_62b7_4d36_bbfb_a031aaa8f782.slice/crio-a9dc41e255780fb4f060abf904f47761a6d90fb7cac8f8b7f520c6dcc5ba6e94 WatchSource:0}: Error finding container a9dc41e255780fb4f060abf904f47761a6d90fb7cac8f8b7f520c6dcc5ba6e94: Status 404 returned error can't find the container with id a9dc41e255780fb4f060abf904f47761a6d90fb7cac8f8b7f520c6dcc5ba6e94
Jul 30 22:46:31.200915 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:31.200873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1780e2f1-70d4-4b00-9a5f-595c19eaee9d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jjs7h\" (UID: \"1780e2f1-70d4-4b00-9a5f-595c19eaee9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:31.203579 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:31.203556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1780e2f1-70d4-4b00-9a5f-595c19eaee9d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jjs7h\" (UID: \"1780e2f1-70d4-4b00-9a5f-595c19eaee9d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:31.486208 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:31.486122    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:31.617994 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:31.617961    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"]
Jul 30 22:46:31.620992 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:46:31.620961    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1780e2f1_70d4_4b00_9a5f_595c19eaee9d.slice/crio-a855d8d1c7229616e7ef77a04c47f308ef65bc610b60f0e858938898c0ffca68 WatchSource:0}: Error finding container a855d8d1c7229616e7ef77a04c47f308ef65bc610b60f0e858938898c0ffca68: Status 404 returned error can't find the container with id a855d8d1c7229616e7ef77a04c47f308ef65bc610b60f0e858938898c0ffca68
Jul 30 22:46:31.660553 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:31.660515    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h" event={"ID":"1780e2f1-70d4-4b00-9a5f-595c19eaee9d","Type":"ContainerStarted","Data":"a855d8d1c7229616e7ef77a04c47f308ef65bc610b60f0e858938898c0ffca68"}
Jul 30 22:46:31.661866 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:31.661838    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w" event={"ID":"7130dbaf-62b7-4d36-bbfb-a031aaa8f782","Type":"ContainerStarted","Data":"a9dc41e255780fb4f060abf904f47761a6d90fb7cac8f8b7f520c6dcc5ba6e94"}
Jul 30 22:46:34.443322 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:34.443283    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"]
Jul 30 22:46:34.443719 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:34.443513    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj" podUID="6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" containerName="discovery" containerID="cri-o://115dcf14cf1bd4d18ae0990792eef0006e667bf581bb8cbc7d4a8b20258c084a" gracePeriod=30
Jul 30 22:46:34.678123 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:34.678086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w" event={"ID":"7130dbaf-62b7-4d36-bbfb-a031aaa8f782","Type":"ContainerStarted","Data":"bc026a73d18a751339e8c57ad36c70f5d344ee011897f9c3c7ada2035f77e62b"}
Jul 30 22:46:34.678305 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:34.678219    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:46:34.679832 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:34.679804    2578 generic.go:356] "Generic (PLEG): container finished" podID="6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" containerID="115dcf14cf1bd4d18ae0990792eef0006e667bf581bb8cbc7d4a8b20258c084a" exitCode=0
Jul 30 22:46:34.679944 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:34.679862    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj" event={"ID":"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0","Type":"ContainerDied","Data":"115dcf14cf1bd4d18ae0990792eef0006e667bf581bb8cbc7d4a8b20258c084a"}
Jul 30 22:46:34.706625 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:34.706518    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w" podStartSLOduration=2.300876395 podStartE2EDuration="4.706502667s" totalImagesPullingTime="2.405626272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:46:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:46:34.704252038 +0000 UTC m=+374.879215717" watchObservedRunningTime="2026-07-30 22:46:34.706502667 +0000 UTC m=+374.881466356"
Jul 30 22:46:35.198876 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.198848    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:35.235086 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.234947    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-cacerts\") pod \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") "
Jul 30 22:46:35.235086 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.234986    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-local-certs\") pod \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") "
Jul 30 22:46:35.235086 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.235013    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-kubeconfig\") pod \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") "
Jul 30 22:46:35.235086 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.235032    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-dns-cert\") pod \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") "
Jul 30 22:46:35.235349 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.235094    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-ca-configmap\") pod \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") "
Jul 30 22:46:35.235349 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.235118    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb9fb\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-kube-api-access-kb9fb\") pod \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") "
Jul 30 22:46:35.235349 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.235181    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-token\") pod \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\" (UID: \"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0\") "
Jul 30 22:46:35.235795 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.235737    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" (UID: "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 22:46:35.237989 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.237942    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" (UID: "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 22:46:35.238117 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.238018    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-local-certs" (OuterVolumeSpecName: "local-certs") pod "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" (UID: "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 22:46:35.238117 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.238025    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-cacerts" (OuterVolumeSpecName: "cacerts") pod "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" (UID: "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 22:46:35.238237 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.238193    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" (UID: "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 22:46:35.238658 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.238630    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-token" (OuterVolumeSpecName: "istio-token") pod "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" (UID: "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:46:35.239125 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.239106    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-kube-api-access-kb9fb" (OuterVolumeSpecName: "kube-api-access-kb9fb") pod "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" (UID: "6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0"). InnerVolumeSpecName "kube-api-access-kb9fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:46:35.335962 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.335930    2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-cacerts\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:46:35.335962 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.335955    2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-local-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:46:35.335962 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.335964    2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-kubeconfig\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:46:35.336231 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.335972    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-dns-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:46:35.336231 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.335981    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-csr-ca-configmap\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:46:35.336231 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.335990    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb9fb\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-kube-api-access-kb9fb\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:46:35.336231 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.335999    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0-istio-token\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:46:35.634312 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.634284    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-5w7xx"
Jul 30 22:46:35.687671 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.687644    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"
Jul 30 22:46:35.687671 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.687661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj" event={"ID":"6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0","Type":"ContainerDied","Data":"b727b4a4e7c4d3bad1b7731c262182a969b45ac28a30dfdf1999105c84ba0b27"}
Jul 30 22:46:35.687880 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.687732    2578 scope.go:117] "RemoveContainer" containerID="115dcf14cf1bd4d18ae0990792eef0006e667bf581bb8cbc7d4a8b20258c084a"
Jul 30 22:46:35.690639 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.690589    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h" event={"ID":"1780e2f1-70d4-4b00-9a5f-595c19eaee9d","Type":"ContainerStarted","Data":"65c6d08d24755cacdd0acb21e5a3767c0de5ec01a299b99918c67e86684cca13"}
Jul 30 22:46:35.690639 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.690637    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:46:35.707762 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.707717    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h" podStartSLOduration=2.096003802 podStartE2EDuration="5.707701699s" totalImagesPullingTime="3.611697897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:46:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:46:35.706750406 +0000 UTC m=+375.881714086" watchObservedRunningTime="2026-07-30 22:46:35.707701699 +0000 UTC m=+375.882665377"
Jul 30 22:46:35.727437 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.724601    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"]
Jul 30 22:46:35.729781 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:35.729751    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-7hkzj"]
Jul 30 22:46:36.340686 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:46:36.340648    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" path="/var/lib/kubelet/pods/6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0/volumes"
Jul 30 22:47:05.695800 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:05.695767    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkz9w"
Jul 30 22:47:06.697620 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:06.697583    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jjs7h"
Jul 30 22:47:36.130031 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.129996    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-qpwsq"]
Jul 30 22:47:36.130507 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.130295    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" containerName="discovery"
Jul 30 22:47:36.130507 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.130306    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" containerName="discovery"
Jul 30 22:47:36.130507 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.130348    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dd52d78-98a2-4f44-863c-8e7a6a9a8fc0" containerName="discovery"
Jul 30 22:47:36.132975 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.132959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq"
Jul 30 22:47:36.137412 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.137383    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 30 22:47:36.137568 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.137555    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 30 22:47:36.137624 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.137595    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ht2bb\""
Jul 30 22:47:36.156557 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.156526    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-qpwsq"]
Jul 30 22:47:36.213407 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.213366    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdm6f\" (UniqueName: \"kubernetes.io/projected/091be627-208c-484a-966b-f56076624fb4-kube-api-access-xdm6f\") pod \"authorino-operator-68d88948d8-qpwsq\" (UID: \"091be627-208c-484a-966b-f56076624fb4\") " pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq"
Jul 30 22:47:36.313982 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.313940    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdm6f\" (UniqueName: \"kubernetes.io/projected/091be627-208c-484a-966b-f56076624fb4-kube-api-access-xdm6f\") pod \"authorino-operator-68d88948d8-qpwsq\" (UID: \"091be627-208c-484a-966b-f56076624fb4\") " pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq"
Jul 30 22:47:36.323476 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.323442    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdm6f\" (UniqueName: \"kubernetes.io/projected/091be627-208c-484a-966b-f56076624fb4-kube-api-access-xdm6f\") pod \"authorino-operator-68d88948d8-qpwsq\" (UID: \"091be627-208c-484a-966b-f56076624fb4\") " pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq"
Jul 30 22:47:36.443157 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.443122    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq"
Jul 30 22:47:36.575636 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.575604    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-qpwsq"]
Jul 30 22:47:36.578815 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:47:36.578793    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091be627_208c_484a_966b_f56076624fb4.slice/crio-b42df9537c3e2b60d5fdbd49f219af11edfda92b3f92ca37ba5597c4c8fb2cff WatchSource:0}: Error finding container b42df9537c3e2b60d5fdbd49f219af11edfda92b3f92ca37ba5597c4c8fb2cff: Status 404 returned error can't find the container with id b42df9537c3e2b60d5fdbd49f219af11edfda92b3f92ca37ba5597c4c8fb2cff
Jul 30 22:47:36.932838 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:36.932743    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq" event={"ID":"091be627-208c-484a-966b-f56076624fb4","Type":"ContainerStarted","Data":"b42df9537c3e2b60d5fdbd49f219af11edfda92b3f92ca37ba5597c4c8fb2cff"}
Jul 30 22:47:38.355086 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:38.355035    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"]
Jul 30 22:47:38.362860 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:38.362837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:47:38.375861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:38.375835    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-k5brr\""
Jul 30 22:47:38.390441 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:38.390414    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"]
Jul 30 22:47:38.431268 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:38.431235    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njvjk\" (UniqueName: \"kubernetes.io/projected/9c7b92bf-b73a-4723-952e-e4b09bdeebfb-kube-api-access-njvjk\") pod \"limitador-operator-controller-manager-6c549b6d85-t49g8\" (UID: \"9c7b92bf-b73a-4723-952e-e4b09bdeebfb\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:47:38.532133 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:38.532100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njvjk\" (UniqueName: \"kubernetes.io/projected/9c7b92bf-b73a-4723-952e-e4b09bdeebfb-kube-api-access-njvjk\") pod \"limitador-operator-controller-manager-6c549b6d85-t49g8\" (UID: \"9c7b92bf-b73a-4723-952e-e4b09bdeebfb\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:47:38.543040 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:38.543006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njvjk\" (UniqueName: \"kubernetes.io/projected/9c7b92bf-b73a-4723-952e-e4b09bdeebfb-kube-api-access-njvjk\") pod \"limitador-operator-controller-manager-6c549b6d85-t49g8\" (UID: \"9c7b92bf-b73a-4723-952e-e4b09bdeebfb\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:47:38.675959 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:38.675866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:47:39.026747 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:39.026721    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"]
Jul 30 22:47:39.031195 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:47:39.031169    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7b92bf_b73a_4723_952e_e4b09bdeebfb.slice/crio-73618336df435f9a4a90d9c0164fc4314cd67211ed825214043d3b38f9366073 WatchSource:0}: Error finding container 73618336df435f9a4a90d9c0164fc4314cd67211ed825214043d3b38f9366073: Status 404 returned error can't find the container with id 73618336df435f9a4a90d9c0164fc4314cd67211ed825214043d3b38f9366073
Jul 30 22:47:39.947987 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:39.947947    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8" event={"ID":"9c7b92bf-b73a-4723-952e-e4b09bdeebfb","Type":"ContainerStarted","Data":"73618336df435f9a4a90d9c0164fc4314cd67211ed825214043d3b38f9366073"}
Jul 30 22:47:39.949713 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:39.949687    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq" event={"ID":"091be627-208c-484a-966b-f56076624fb4","Type":"ContainerStarted","Data":"0c0d616739d6d6d958f9ea6be117f3db57d25f3e7531c5f6eef972a8bd9f87d2"}
Jul 30 22:47:39.949863 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:39.949846    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq"
Jul 30 22:47:39.968688 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:39.968642    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq" podStartSLOduration=1.597554318 podStartE2EDuration="3.968627578s" totalImagesPullingTime="2.37107326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:39.967500702 +0000 UTC m=+440.142464381" watchObservedRunningTime="2026-07-30 22:47:39.968627578 +0000 UTC m=+440.143591256"
Jul 30 22:47:40.661720 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.661683    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"]
Jul 30 22:47:40.665151 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.665125    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"
Jul 30 22:47:40.672176 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.672145    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 30 22:47:40.675977 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.675952    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-pt45p\""
Jul 30 22:47:40.692834 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.692748    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"]
Jul 30 22:47:40.750339 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.750300    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5v26\" (UniqueName: \"kubernetes.io/projected/c2deeb6e-cd26-4ffd-9eed-f830be5d6c77-kube-api-access-v5v26\") pod \"dns-operator-controller-manager-5985564776-6d7qw\" (UID: \"c2deeb6e-cd26-4ffd-9eed-f830be5d6c77\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"
Jul 30 22:47:40.851405 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.851378    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5v26\" (UniqueName: \"kubernetes.io/projected/c2deeb6e-cd26-4ffd-9eed-f830be5d6c77-kube-api-access-v5v26\") pod \"dns-operator-controller-manager-5985564776-6d7qw\" (UID: \"c2deeb6e-cd26-4ffd-9eed-f830be5d6c77\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"
Jul 30 22:47:40.861992 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.861971    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5v26\" (UniqueName: \"kubernetes.io/projected/c2deeb6e-cd26-4ffd-9eed-f830be5d6c77-kube-api-access-v5v26\") pod \"dns-operator-controller-manager-5985564776-6d7qw\" (UID: \"c2deeb6e-cd26-4ffd-9eed-f830be5d6c77\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"
Jul 30 22:47:40.955246 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.955206    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8" event={"ID":"9c7b92bf-b73a-4723-952e-e4b09bdeebfb","Type":"ContainerStarted","Data":"873ae95fb5988e6ea0ec1c9725eb5a35f59d5d8952b66a7de2298649fb9577e3"}
Jul 30 22:47:40.955717 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.955311    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:47:40.977176 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.977147    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"
Jul 30 22:47:40.983711 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:40.983663    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8" podStartSLOduration=1.199436821 podStartE2EDuration="2.983647965s" totalImagesPullingTime="1.784211144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:40.981949536 +0000 UTC m=+441.156913214" watchObservedRunningTime="2026-07-30 22:47:40.983647965 +0000 UTC m=+441.158611645"
Jul 30 22:47:41.113406 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:41.113375    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"]
Jul 30 22:47:41.116388 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:47:41.116355    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2deeb6e_cd26_4ffd_9eed_f830be5d6c77.slice/crio-ce9c551bf6b5aacfc6f9ed5eb15fe5f343f6dc8fa82ec2a8a1401af6cc2b6384 WatchSource:0}: Error finding container ce9c551bf6b5aacfc6f9ed5eb15fe5f343f6dc8fa82ec2a8a1401af6cc2b6384: Status 404 returned error can't find the container with id ce9c551bf6b5aacfc6f9ed5eb15fe5f343f6dc8fa82ec2a8a1401af6cc2b6384
Jul 30 22:47:41.961805 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:41.961766    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw" event={"ID":"c2deeb6e-cd26-4ffd-9eed-f830be5d6c77","Type":"ContainerStarted","Data":"ce9c551bf6b5aacfc6f9ed5eb15fe5f343f6dc8fa82ec2a8a1401af6cc2b6384"}
Jul 30 22:47:43.970956 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:43.970917    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw" event={"ID":"c2deeb6e-cd26-4ffd-9eed-f830be5d6c77","Type":"ContainerStarted","Data":"911edd1224ab6d31c16fc2249769ef0055b670e15493e7a023ff5eb356762b49"}
Jul 30 22:47:43.971342 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:43.970983    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"
Jul 30 22:47:43.990943 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:43.990898    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw" podStartSLOduration=2.192665872 podStartE2EDuration="3.990882465s" totalImagesPullingTime="1.798216593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:43.990513295 +0000 UTC m=+444.165476973" watchObservedRunningTime="2026-07-30 22:47:43.990882465 +0000 UTC m=+444.165846145"
Jul 30 22:47:44.755353 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.755308    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"]
Jul 30 22:47:44.758333 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.758317    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:47:44.760730 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.760706    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9q7ms\""
Jul 30 22:47:44.773541 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.773515    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"]
Jul 30 22:47:44.780249 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.780218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v5cd\" (UniqueName: \"kubernetes.io/projected/f60c548d-6623-4c5d-8e91-4314b7264ee8-kube-api-access-4v5cd\") pod \"kuadrant-operator-controller-manager-58fc56757c-44pss\" (UID: \"f60c548d-6623-4c5d-8e91-4314b7264ee8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:47:44.780387 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.780352    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f60c548d-6623-4c5d-8e91-4314b7264ee8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-44pss\" (UID: \"f60c548d-6623-4c5d-8e91-4314b7264ee8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:47:44.881534 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.881496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f60c548d-6623-4c5d-8e91-4314b7264ee8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-44pss\" (UID: \"f60c548d-6623-4c5d-8e91-4314b7264ee8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:47:44.881707 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.881548    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v5cd\" (UniqueName: \"kubernetes.io/projected/f60c548d-6623-4c5d-8e91-4314b7264ee8-kube-api-access-4v5cd\") pod \"kuadrant-operator-controller-manager-58fc56757c-44pss\" (UID: \"f60c548d-6623-4c5d-8e91-4314b7264ee8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:47:44.881898 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.881879    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f60c548d-6623-4c5d-8e91-4314b7264ee8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-44pss\" (UID: \"f60c548d-6623-4c5d-8e91-4314b7264ee8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:47:44.893150 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:44.893122    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v5cd\" (UniqueName: \"kubernetes.io/projected/f60c548d-6623-4c5d-8e91-4314b7264ee8-kube-api-access-4v5cd\") pod \"kuadrant-operator-controller-manager-58fc56757c-44pss\" (UID: \"f60c548d-6623-4c5d-8e91-4314b7264ee8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:47:45.068240 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:45.068139    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:47:45.204094 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:45.204039    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"]
Jul 30 22:47:45.207631 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:47:45.207604    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf60c548d_6623_4c5d_8e91_4314b7264ee8.slice/crio-80bc7257b3099a048703b8d86e0d56b5ecf658d988a63d51a3fc279304c110a7 WatchSource:0}: Error finding container 80bc7257b3099a048703b8d86e0d56b5ecf658d988a63d51a3fc279304c110a7: Status 404 returned error can't find the container with id 80bc7257b3099a048703b8d86e0d56b5ecf658d988a63d51a3fc279304c110a7
Jul 30 22:47:45.980623 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:45.980589    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss" event={"ID":"f60c548d-6623-4c5d-8e91-4314b7264ee8","Type":"ContainerStarted","Data":"80bc7257b3099a048703b8d86e0d56b5ecf658d988a63d51a3fc279304c110a7"}
Jul 30 22:47:50.002875 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:50.002830    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss" event={"ID":"f60c548d-6623-4c5d-8e91-4314b7264ee8","Type":"ContainerStarted","Data":"e6f7611f307cd435d16048586d08f253b43a1077963818f8c426db87cea28c76"}
Jul 30 22:47:50.003279 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:50.002975    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:47:50.026111 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:50.026043    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss" podStartSLOduration=2.1256788110000002 podStartE2EDuration="6.026024563s" totalImagesPullingTime="3.900345752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:50.024154217 +0000 UTC m=+450.199117908" watchObservedRunningTime="2026-07-30 22:47:50.026024563 +0000 UTC m=+450.200988241"
Jul 30 22:47:50.957894 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:50.957862    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-qpwsq"
Jul 30 22:47:51.916861 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:51.916826    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"]
Jul 30 22:47:51.920251 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:51.920224    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:51.922771 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:51.922748    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 30 22:47:51.922892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:51.922882    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-x69mt\""
Jul 30 22:47:51.922964 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:51.922951    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 30 22:47:51.927986 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:51.927963    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"]
Jul 30 22:47:51.964237 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:51.964205    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:47:52.042521 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.042492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b354f0-4267-46c8-a78c-86f2fcb5425f-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-xwqsq\" (UID: \"80b354f0-4267-46c8-a78c-86f2fcb5425f\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.042684 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.042534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjl5w\" (UniqueName: \"kubernetes.io/projected/80b354f0-4267-46c8-a78c-86f2fcb5425f-kube-api-access-jjl5w\") pod \"kuadrant-console-plugin-598f86468b-xwqsq\" (UID: \"80b354f0-4267-46c8-a78c-86f2fcb5425f\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.042684 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.042561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/80b354f0-4267-46c8-a78c-86f2fcb5425f-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-xwqsq\" (UID: \"80b354f0-4267-46c8-a78c-86f2fcb5425f\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.143690 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.143647    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/80b354f0-4267-46c8-a78c-86f2fcb5425f-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-xwqsq\" (UID: \"80b354f0-4267-46c8-a78c-86f2fcb5425f\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.143889 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.143738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b354f0-4267-46c8-a78c-86f2fcb5425f-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-xwqsq\" (UID: \"80b354f0-4267-46c8-a78c-86f2fcb5425f\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.143889 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.143781    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjl5w\" (UniqueName: \"kubernetes.io/projected/80b354f0-4267-46c8-a78c-86f2fcb5425f-kube-api-access-jjl5w\") pod \"kuadrant-console-plugin-598f86468b-xwqsq\" (UID: \"80b354f0-4267-46c8-a78c-86f2fcb5425f\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.144317 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.144295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/80b354f0-4267-46c8-a78c-86f2fcb5425f-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-xwqsq\" (UID: \"80b354f0-4267-46c8-a78c-86f2fcb5425f\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.146261 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.146241    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/80b354f0-4267-46c8-a78c-86f2fcb5425f-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-xwqsq\" (UID: \"80b354f0-4267-46c8-a78c-86f2fcb5425f\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.153711 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.153683    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjl5w\" (UniqueName: \"kubernetes.io/projected/80b354f0-4267-46c8-a78c-86f2fcb5425f-kube-api-access-jjl5w\") pod \"kuadrant-console-plugin-598f86468b-xwqsq\" (UID: \"80b354f0-4267-46c8-a78c-86f2fcb5425f\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.229434 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.229397    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"
Jul 30 22:47:52.357637 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:52.357605    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq"]
Jul 30 22:47:52.360458 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:47:52.360423    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80b354f0_4267_46c8_a78c_86f2fcb5425f.slice/crio-c4994e4679b9fc290c1e4b69feba74f703d8cfa22490829c584bcf6f39a332ce WatchSource:0}: Error finding container c4994e4679b9fc290c1e4b69feba74f703d8cfa22490829c584bcf6f39a332ce: Status 404 returned error can't find the container with id c4994e4679b9fc290c1e4b69feba74f703d8cfa22490829c584bcf6f39a332ce
Jul 30 22:47:53.017339 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:53.017308    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq" event={"ID":"80b354f0-4267-46c8-a78c-86f2fcb5425f","Type":"ContainerStarted","Data":"c4994e4679b9fc290c1e4b69feba74f703d8cfa22490829c584bcf6f39a332ce"}
Jul 30 22:47:54.977687 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:54.977658    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-6d7qw"
Jul 30 22:47:58.042187 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:58.042150    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq" event={"ID":"80b354f0-4267-46c8-a78c-86f2fcb5425f","Type":"ContainerStarted","Data":"200cf11e7c645e037cb94ab2c7e48d1c13175535643c6e2eda31c3ce8f5e27fa"}
Jul 30 22:47:58.063892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:47:58.063840    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-xwqsq" podStartSLOduration=2.426263108 podStartE2EDuration="7.063824224s" totalImagesPullingTime="4.637561116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:47:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:47:58.06232978 +0000 UTC m=+458.237293459" watchObservedRunningTime="2026-07-30 22:47:58.063824224 +0000 UTC m=+458.238787904"
Jul 30 22:48:01.010270 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:01.010237    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:02.173976 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.173942    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"]
Jul 30 22:48:02.178237 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.178211    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:02.191903 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.191869    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"]
Jul 30 22:48:02.232948 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.232914    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7a0bdb0-98a8-475a-8891-c47e4a400398-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" (UID: \"b7a0bdb0-98a8-475a-8891-c47e4a400398\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:02.233165 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.233002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tcvx\" (UniqueName: \"kubernetes.io/projected/b7a0bdb0-98a8-475a-8891-c47e4a400398-kube-api-access-4tcvx\") pod \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" (UID: \"b7a0bdb0-98a8-475a-8891-c47e4a400398\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:02.334330 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.334296    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tcvx\" (UniqueName: \"kubernetes.io/projected/b7a0bdb0-98a8-475a-8891-c47e4a400398-kube-api-access-4tcvx\") pod \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" (UID: \"b7a0bdb0-98a8-475a-8891-c47e4a400398\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:02.334330 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.334333    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7a0bdb0-98a8-475a-8891-c47e4a400398-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" (UID: \"b7a0bdb0-98a8-475a-8891-c47e4a400398\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:02.334689 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.334673    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7a0bdb0-98a8-475a-8891-c47e4a400398-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" (UID: \"b7a0bdb0-98a8-475a-8891-c47e4a400398\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:02.345458 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.345420    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tcvx\" (UniqueName: \"kubernetes.io/projected/b7a0bdb0-98a8-475a-8891-c47e4a400398-kube-api-access-4tcvx\") pod \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" (UID: \"b7a0bdb0-98a8-475a-8891-c47e4a400398\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:02.488585 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.488542    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:02.640747 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.640721    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"]
Jul 30 22:48:02.643350 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:48:02.643311    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a0bdb0_98a8_475a_8891_c47e4a400398.slice/crio-24577daa866edc4b0c7eb57366146b26f65e7ad10712ca946467f88916983024 WatchSource:0}: Error finding container 24577daa866edc4b0c7eb57366146b26f65e7ad10712ca946467f88916983024: Status 404 returned error can't find the container with id 24577daa866edc4b0c7eb57366146b26f65e7ad10712ca946467f88916983024
Jul 30 22:48:02.865878 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.865778    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"]
Jul 30 22:48:02.866104 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.866073    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" containerName="manager" containerID="cri-o://e6f7611f307cd435d16048586d08f253b43a1077963818f8c426db87cea28c76" gracePeriod=2
Jul 30 22:48:02.876204 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:02.876150    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f60c548d-6623-4c5d-8e91-4314b7264ee8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-58fc56757c-44pss\": pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:02.877582 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.877553    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"]
Jul 30 22:48:02.890134 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.890028    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"]
Jul 30 22:48:02.892021 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.891982    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"]
Jul 30 22:48:02.892460 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.892444    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" containerName="manager"
Jul 30 22:48:02.892507 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.892462    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" containerName="manager"
Jul 30 22:48:02.892540 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.892520    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" containerName="manager"
Jul 30 22:48:02.895412 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.895387    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:02.900800 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.900768    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"]
Jul 30 22:48:02.911426 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.911392    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"]
Jul 30 22:48:02.921769 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.921732    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"]
Jul 30 22:48:02.921961 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.921941    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" containerName="manager" containerID="cri-o://873ae95fb5988e6ea0ec1c9725eb5a35f59d5d8952b66a7de2298649fb9577e3" gracePeriod=2
Jul 30 22:48:02.937455 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.937421    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"]
Jul 30 22:48:02.939651 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.939616    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"]
Jul 30 22:48:02.940551 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.940144    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" containerName="manager"
Jul 30 22:48:02.940551 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.940174    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" containerName="manager"
Jul 30 22:48:02.940551 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.940201    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" containerName="manager"
Jul 30 22:48:02.940551 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.940216    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" containerName="manager"
Jul 30 22:48:02.940551 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.940355    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" containerName="manager"
Jul 30 22:48:02.940551 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.940367    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" containerName="manager"
Jul 30 22:48:02.943474 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.943415    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:02.943821 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:02.943787    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:02.946954 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:02.946911    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:02.951373 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:02.951340    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-t49g8\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:02.957664 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.957636    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"]
Jul 30 22:48:02.961204 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.961173    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"]
Jul 30 22:48:02.961378 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.961325    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"
Jul 30 22:48:02.973556 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:02.973515    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"]
Jul 30 22:48:02.997420 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:02.997382    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:02.999545 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:02.999514    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-t49g8\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:03.039420 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.039384    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69cc1ef8-1338-416f-9ad8-938dddf96cdf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-vtjs8\" (UID: \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:03.039591 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.039439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5flk\" (UniqueName: \"kubernetes.io/projected/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-kube-api-access-f5flk\") pod \"kuadrant-operator-controller-manager-58fc56757c-vrcmr\" (UID: \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:03.039591 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.039546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8prg\" (UniqueName: \"kubernetes.io/projected/69cc1ef8-1338-416f-9ad8-938dddf96cdf-kube-api-access-q8prg\") pod \"kuadrant-operator-controller-manager-96db98c47-vtjs8\" (UID: \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:03.039684 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.039616    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vrcmr\" (UID: \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:03.039684 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.039652    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bppr7\" (UniqueName: \"kubernetes.io/projected/0c8e620f-9826-4761-9ab2-e99a790ce61e-kube-api-access-bppr7\") pod \"limitador-operator-controller-manager-6c549b6d85-nfs8s\" (UID: \"0c8e620f-9826-4761-9ab2-e99a790ce61e\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"
Jul 30 22:48:03.066808 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.066768    2578 generic.go:356] "Generic (PLEG): container finished" podID="f60c548d-6623-4c5d-8e91-4314b7264ee8" containerID="e6f7611f307cd435d16048586d08f253b43a1077963818f8c426db87cea28c76" exitCode=0
Jul 30 22:48:03.069102 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.068966    2578 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx" container="manager"
Jul 30 22:48:03.070702 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.070639    2578 generic.go:356] "Generic (PLEG): container finished" podID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" containerID="873ae95fb5988e6ea0ec1c9725eb5a35f59d5d8952b66a7de2298649fb9577e3" exitCode=0
Jul 30 22:48:03.072418 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.072378    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-t49g8\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:03.075126 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.075044    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:03.077275 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.077242    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:03.140951 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.140855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5flk\" (UniqueName: \"kubernetes.io/projected/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-kube-api-access-f5flk\") pod \"kuadrant-operator-controller-manager-58fc56757c-vrcmr\" (UID: \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:03.140951 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.140918    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8prg\" (UniqueName: \"kubernetes.io/projected/69cc1ef8-1338-416f-9ad8-938dddf96cdf-kube-api-access-q8prg\") pod \"kuadrant-operator-controller-manager-96db98c47-vtjs8\" (UID: \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:03.141199 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.140969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vrcmr\" (UID: \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:03.141199 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.141003    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bppr7\" (UniqueName: \"kubernetes.io/projected/0c8e620f-9826-4761-9ab2-e99a790ce61e-kube-api-access-bppr7\") pod \"limitador-operator-controller-manager-6c549b6d85-nfs8s\" (UID: \"0c8e620f-9826-4761-9ab2-e99a790ce61e\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"
Jul 30 22:48:03.141199 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.141095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69cc1ef8-1338-416f-9ad8-938dddf96cdf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-vtjs8\" (UID: \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:03.141515 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.141493    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69cc1ef8-1338-416f-9ad8-938dddf96cdf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-96db98c47-vtjs8\" (UID: \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:03.141554 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.141506    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-vrcmr\" (UID: \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:03.152191 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.152158    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8prg\" (UniqueName: \"kubernetes.io/projected/69cc1ef8-1338-416f-9ad8-938dddf96cdf-kube-api-access-q8prg\") pod \"kuadrant-operator-controller-manager-96db98c47-vtjs8\" (UID: \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:03.152932 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.152907    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5flk\" (UniqueName: \"kubernetes.io/projected/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-kube-api-access-f5flk\") pod \"kuadrant-operator-controller-manager-58fc56757c-vrcmr\" (UID: \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:03.153098 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.153081    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bppr7\" (UniqueName: \"kubernetes.io/projected/0c8e620f-9826-4761-9ab2-e99a790ce61e-kube-api-access-bppr7\") pod \"limitador-operator-controller-manager-6c549b6d85-nfs8s\" (UID: \"0c8e620f-9826-4761-9ab2-e99a790ce61e\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"
Jul 30 22:48:03.162267 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.162243    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:03.164892 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.164853    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-t49g8\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:03.167077 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.167023    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:03.169276 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.169246    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:03.169863 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.169847    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:03.172219 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.172191    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:03.174325 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.174300    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:03.176504 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:03.176480    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-t49g8\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:03.241547 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.241509    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f60c548d-6623-4c5d-8e91-4314b7264ee8-extensions-socket-volume\") pod \"f60c548d-6623-4c5d-8e91-4314b7264ee8\" (UID: \"f60c548d-6623-4c5d-8e91-4314b7264ee8\") "
Jul 30 22:48:03.241657 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.241613    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njvjk\" (UniqueName: \"kubernetes.io/projected/9c7b92bf-b73a-4723-952e-e4b09bdeebfb-kube-api-access-njvjk\") pod \"9c7b92bf-b73a-4723-952e-e4b09bdeebfb\" (UID: \"9c7b92bf-b73a-4723-952e-e4b09bdeebfb\") "
Jul 30 22:48:03.241714 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.241658    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v5cd\" (UniqueName: \"kubernetes.io/projected/f60c548d-6623-4c5d-8e91-4314b7264ee8-kube-api-access-4v5cd\") pod \"f60c548d-6623-4c5d-8e91-4314b7264ee8\" (UID: \"f60c548d-6623-4c5d-8e91-4314b7264ee8\") "
Jul 30 22:48:03.242093 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.242039    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f60c548d-6623-4c5d-8e91-4314b7264ee8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f60c548d-6623-4c5d-8e91-4314b7264ee8" (UID: "f60c548d-6623-4c5d-8e91-4314b7264ee8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 22:48:03.244101 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.244068    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c7b92bf-b73a-4723-952e-e4b09bdeebfb-kube-api-access-njvjk" (OuterVolumeSpecName: "kube-api-access-njvjk") pod "9c7b92bf-b73a-4723-952e-e4b09bdeebfb" (UID: "9c7b92bf-b73a-4723-952e-e4b09bdeebfb"). InnerVolumeSpecName "kube-api-access-njvjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:48:03.244212 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.244068    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f60c548d-6623-4c5d-8e91-4314b7264ee8-kube-api-access-4v5cd" (OuterVolumeSpecName: "kube-api-access-4v5cd") pod "f60c548d-6623-4c5d-8e91-4314b7264ee8" (UID: "f60c548d-6623-4c5d-8e91-4314b7264ee8"). InnerVolumeSpecName "kube-api-access-4v5cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:48:03.289958 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.289918    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:03.298735 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.298705    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:03.305584 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.305554    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"
Jul 30 22:48:03.343071 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.343015    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njvjk\" (UniqueName: \"kubernetes.io/projected/9c7b92bf-b73a-4723-952e-e4b09bdeebfb-kube-api-access-njvjk\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:03.343071 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.343065    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v5cd\" (UniqueName: \"kubernetes.io/projected/f60c548d-6623-4c5d-8e91-4314b7264ee8-kube-api-access-4v5cd\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:03.343071 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.343080    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f60c548d-6623-4c5d-8e91-4314b7264ee8-extensions-socket-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:03.470153 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.470122    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"]
Jul 30 22:48:03.474274 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:48:03.474244    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69cc1ef8_1338_416f_9ad8_938dddf96cdf.slice/crio-76247f9e5902932fd4cf841823b6490a25b6ea4829328b991cfcf2d001ae9b9f WatchSource:0}: Error finding container 76247f9e5902932fd4cf841823b6490a25b6ea4829328b991cfcf2d001ae9b9f: Status 404 returned error can't find the container with id 76247f9e5902932fd4cf841823b6490a25b6ea4829328b991cfcf2d001ae9b9f
Jul 30 22:48:03.491017 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.490990    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"]
Jul 30 22:48:03.493537 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:48:03.493511    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525e6b68_7b2b_4ad9_9a8a_526c63dbbac6.slice/crio-30b3a2b992a2afa87ea9763e0a952b1f8f6e97c08d15d6d7096fc64a131d5527 WatchSource:0}: Error finding container 30b3a2b992a2afa87ea9763e0a952b1f8f6e97c08d15d6d7096fc64a131d5527: Status 404 returned error can't find the container with id 30b3a2b992a2afa87ea9763e0a952b1f8f6e97c08d15d6d7096fc64a131d5527
Jul 30 22:48:03.508713 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:03.508679    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"]
Jul 30 22:48:03.520288 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:48:03.520249    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c8e620f_9826_4761_9ab2_e99a790ce61e.slice/crio-e1728f3d5d1411605a6ed6606c2e9d7b129a8c4d87e494684dd541dc55c89130 WatchSource:0}: Error finding container e1728f3d5d1411605a6ed6606c2e9d7b129a8c4d87e494684dd541dc55c89130: Status 404 returned error can't find the container with id e1728f3d5d1411605a6ed6606c2e9d7b129a8c4d87e494684dd541dc55c89130
Jul 30 22:48:04.078250 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.078218    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:04.078250 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.078249    2578 scope.go:117] "RemoveContainer" containerID="873ae95fb5988e6ea0ec1c9725eb5a35f59d5d8952b66a7de2298649fb9577e3"
Jul 30 22:48:04.080253 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.080221    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s" event={"ID":"0c8e620f-9826-4761-9ab2-e99a790ce61e","Type":"ContainerStarted","Data":"ae7929b3881924e6ff68e3f471c4d65b496d5693e0f7e4bd1de152dee94989b1"}
Jul 30 22:48:04.080399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.080299    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s" event={"ID":"0c8e620f-9826-4761-9ab2-e99a790ce61e","Type":"ContainerStarted","Data":"e1728f3d5d1411605a6ed6606c2e9d7b129a8c4d87e494684dd541dc55c89130"}
Jul 30 22:48:04.080399 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.080333    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"
Jul 30 22:48:04.081002 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.080971    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-t49g8\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:04.082274 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.082251    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr" event={"ID":"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6","Type":"ContainerStarted","Data":"4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2"}
Jul 30 22:48:04.082377 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.082308    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:04.082377 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.082332    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr" event={"ID":"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6","Type":"ContainerStarted","Data":"30b3a2b992a2afa87ea9763e0a952b1f8f6e97c08d15d6d7096fc64a131d5527"}
Jul 30 22:48:04.083196 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.083171    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:04.083960 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.083850    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:04.085639 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.085610    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:04.085976 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.085942    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" containerName="manager" containerID="cri-o://62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837" gracePeriod=2
Jul 30 22:48:04.088723 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.088343    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8" event={"ID":"69cc1ef8-1338-416f-9ad8-938dddf96cdf","Type":"ContainerStarted","Data":"7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a"}
Jul 30 22:48:04.088723 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.088393    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8" event={"ID":"69cc1ef8-1338-416f-9ad8-938dddf96cdf","Type":"ContainerStarted","Data":"76247f9e5902932fd4cf841823b6490a25b6ea4829328b991cfcf2d001ae9b9f"}
Jul 30 22:48:04.088723 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.088435    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:04.089209 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.089197    2578 scope.go:117] "RemoveContainer" containerID="e6f7611f307cd435d16048586d08f253b43a1077963818f8c426db87cea28c76"
Jul 30 22:48:04.109709 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.109643    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr" podStartSLOduration=2.109627054 podStartE2EDuration="2.109627054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:04.107823888 +0000 UTC m=+464.282787566" watchObservedRunningTime="2026-07-30 22:48:04.109627054 +0000 UTC m=+464.284590733"
Jul 30 22:48:04.110233 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.110196    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:04.130952 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.130892    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s" podStartSLOduration=2.130874956 podStartE2EDuration="2.130874956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:04.130287993 +0000 UTC m=+464.305251671" watchObservedRunningTime="2026-07-30 22:48:04.130874956 +0000 UTC m=+464.305838634"
Jul 30 22:48:04.133894 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.133863    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:04.137129 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.137101    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-t49g8\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:04.139254 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.139223    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-t49g8\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:04.166903 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.166850    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8" podStartSLOduration=2.166833064 podStartE2EDuration="2.166833064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:04.165323487 +0000 UTC m=+464.340287188" watchObservedRunningTime="2026-07-30 22:48:04.166833064 +0000 UTC m=+464.341796743"
Jul 30 22:48:04.168219 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.168181    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:04.170505 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.170477    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:04.325027 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.324997    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:04.327480 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.327444    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-t49g8\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-t49g8"
Jul 30 22:48:04.329616 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.329552    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:04.331505 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:04.331478    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-44pss\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-44pss"
Jul 30 22:48:04.343266 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.343230    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c7b92bf-b73a-4723-952e-e4b09bdeebfb" path="/var/lib/kubelet/pods/9c7b92bf-b73a-4723-952e-e4b09bdeebfb/volumes"
Jul 30 22:48:04.343987 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.343964    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f60c548d-6623-4c5d-8e91-4314b7264ee8" path="/var/lib/kubelet/pods/f60c548d-6623-4c5d-8e91-4314b7264ee8/volumes"
Jul 30 22:48:04.452529 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.452495    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7a0bdb0-98a8-475a-8891-c47e4a400398-extensions-socket-volume\") pod \"b7a0bdb0-98a8-475a-8891-c47e4a400398\" (UID: \"b7a0bdb0-98a8-475a-8891-c47e4a400398\") "
Jul 30 22:48:04.452716 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.452561    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tcvx\" (UniqueName: \"kubernetes.io/projected/b7a0bdb0-98a8-475a-8891-c47e4a400398-kube-api-access-4tcvx\") pod \"b7a0bdb0-98a8-475a-8891-c47e4a400398\" (UID: \"b7a0bdb0-98a8-475a-8891-c47e4a400398\") "
Jul 30 22:48:04.452808 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.452780    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a0bdb0-98a8-475a-8891-c47e4a400398-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b7a0bdb0-98a8-475a-8891-c47e4a400398" (UID: "b7a0bdb0-98a8-475a-8891-c47e4a400398"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 22:48:04.454920 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.454892    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7a0bdb0-98a8-475a-8891-c47e4a400398-kube-api-access-4tcvx" (OuterVolumeSpecName: "kube-api-access-4tcvx") pod "b7a0bdb0-98a8-475a-8891-c47e4a400398" (UID: "b7a0bdb0-98a8-475a-8891-c47e4a400398"). InnerVolumeSpecName "kube-api-access-4tcvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:48:04.553568 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.553533    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b7a0bdb0-98a8-475a-8891-c47e4a400398-extensions-socket-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:04.553568 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:04.553564    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tcvx\" (UniqueName: \"kubernetes.io/projected/b7a0bdb0-98a8-475a-8891-c47e4a400398-kube-api-access-4tcvx\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:05.095108 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:05.095073    2578 generic.go:356] "Generic (PLEG): container finished" podID="b7a0bdb0-98a8-475a-8891-c47e4a400398" containerID="62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837" exitCode=2
Jul 30 22:48:05.095108 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:05.095109    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:05.095343 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:05.095164    2578 scope.go:117] "RemoveContainer" containerID="62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837"
Jul 30 22:48:05.104121 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:05.104099    2578 scope.go:117] "RemoveContainer" containerID="62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837"
Jul 30 22:48:05.104423 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:05.104404    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837\": container with ID starting with 62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837 not found: ID does not exist" containerID="62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837"
Jul 30 22:48:05.104477 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:05.104432    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837"} err="failed to get container status \"62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837\": rpc error: code = NotFound desc = could not find container \"62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837\": container with ID starting with 62680ca70ba52e109caf247422980c4473cf2094f9c36fdba1f4be41b0812837 not found: ID does not exist"
Jul 30 22:48:05.107294 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:05.107265    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-96db98c47-5dvgx\" is forbidden: User \"system:node:ip-10-0-142-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-225.ec2.internal' and this object" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-5dvgx"
Jul 30 22:48:06.342299 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:06.342268    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7a0bdb0-98a8-475a-8891-c47e4a400398" path="/var/lib/kubelet/pods/b7a0bdb0-98a8-475a-8891-c47e4a400398/volumes"
Jul 30 22:48:15.098288 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.098252    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:15.098812 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.098335    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:15.098812 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.098496    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-nfs8s"
Jul 30 22:48:15.167895 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.167862    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"]
Jul 30 22:48:15.168111 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.168081    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr" podUID="525e6b68-7b2b-4ad9-9a8a-526c63dbbac6" containerName="manager" containerID="cri-o://4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2" gracePeriod=10
Jul 30 22:48:15.416671 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.416640    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:15.542264 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.542238    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5flk\" (UniqueName: \"kubernetes.io/projected/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-kube-api-access-f5flk\") pod \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\" (UID: \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\") "
Jul 30 22:48:15.542444 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.542277    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-extensions-socket-volume\") pod \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\" (UID: \"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6\") "
Jul 30 22:48:15.542700 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.542675    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "525e6b68-7b2b-4ad9-9a8a-526c63dbbac6" (UID: "525e6b68-7b2b-4ad9-9a8a-526c63dbbac6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 22:48:15.544345 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.544318    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-kube-api-access-f5flk" (OuterVolumeSpecName: "kube-api-access-f5flk") pod "525e6b68-7b2b-4ad9-9a8a-526c63dbbac6" (UID: "525e6b68-7b2b-4ad9-9a8a-526c63dbbac6"). InnerVolumeSpecName "kube-api-access-f5flk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:48:15.583709 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.583675    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"]
Jul 30 22:48:15.583974 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.583963    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="525e6b68-7b2b-4ad9-9a8a-526c63dbbac6" containerName="manager"
Jul 30 22:48:15.584019 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.583976    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="525e6b68-7b2b-4ad9-9a8a-526c63dbbac6" containerName="manager"
Jul 30 22:48:15.584079 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.584038    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="525e6b68-7b2b-4ad9-9a8a-526c63dbbac6" containerName="manager"
Jul 30 22:48:15.586844 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.586827    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:15.610020 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.609994    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"]
Jul 30 22:48:15.642970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.642944    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5flk\" (UniqueName: \"kubernetes.io/projected/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-kube-api-access-f5flk\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:15.642970 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.642966    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6-extensions-socket-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:15.743740 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.743704    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ec5f7e-acea-4304-a24c-4ec967075fc9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-jwnr7\" (UID: \"12ec5f7e-acea-4304-a24c-4ec967075fc9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:15.743935 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.743768    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcj4m\" (UniqueName: \"kubernetes.io/projected/12ec5f7e-acea-4304-a24c-4ec967075fc9-kube-api-access-mcj4m\") pod \"kuadrant-operator-controller-manager-68cf589589-jwnr7\" (UID: \"12ec5f7e-acea-4304-a24c-4ec967075fc9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:15.844574 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.844536    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcj4m\" (UniqueName: \"kubernetes.io/projected/12ec5f7e-acea-4304-a24c-4ec967075fc9-kube-api-access-mcj4m\") pod \"kuadrant-operator-controller-manager-68cf589589-jwnr7\" (UID: \"12ec5f7e-acea-4304-a24c-4ec967075fc9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:15.844777 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.844631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ec5f7e-acea-4304-a24c-4ec967075fc9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-jwnr7\" (UID: \"12ec5f7e-acea-4304-a24c-4ec967075fc9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:15.845012 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.844990    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ec5f7e-acea-4304-a24c-4ec967075fc9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-jwnr7\" (UID: \"12ec5f7e-acea-4304-a24c-4ec967075fc9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:15.857767 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.857729    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcj4m\" (UniqueName: \"kubernetes.io/projected/12ec5f7e-acea-4304-a24c-4ec967075fc9-kube-api-access-mcj4m\") pod \"kuadrant-operator-controller-manager-68cf589589-jwnr7\" (UID: \"12ec5f7e-acea-4304-a24c-4ec967075fc9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:15.896804 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:15.896764    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:16.033985 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.033900    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"]
Jul 30 22:48:16.037360 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:48:16.037333    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ec5f7e_acea_4304_a24c_4ec967075fc9.slice/crio-8bec6177bd601e54698c3a0c83f4fa181818e044d9ba5cdc88679cab32f7aa19 WatchSource:0}: Error finding container 8bec6177bd601e54698c3a0c83f4fa181818e044d9ba5cdc88679cab32f7aa19: Status 404 returned error can't find the container with id 8bec6177bd601e54698c3a0c83f4fa181818e044d9ba5cdc88679cab32f7aa19
Jul 30 22:48:16.155711 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.155670    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7" event={"ID":"12ec5f7e-acea-4304-a24c-4ec967075fc9","Type":"ContainerStarted","Data":"23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e"}
Jul 30 22:48:16.156151 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.155891    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:16.156151 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.155934    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7" event={"ID":"12ec5f7e-acea-4304-a24c-4ec967075fc9","Type":"ContainerStarted","Data":"8bec6177bd601e54698c3a0c83f4fa181818e044d9ba5cdc88679cab32f7aa19"}
Jul 30 22:48:16.157258 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.157236    2578 generic.go:356] "Generic (PLEG): container finished" podID="525e6b68-7b2b-4ad9-9a8a-526c63dbbac6" containerID="4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2" exitCode=0
Jul 30 22:48:16.157357 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.157297    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr" event={"ID":"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6","Type":"ContainerDied","Data":"4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2"}
Jul 30 22:48:16.157357 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.157328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr" event={"ID":"525e6b68-7b2b-4ad9-9a8a-526c63dbbac6","Type":"ContainerDied","Data":"30b3a2b992a2afa87ea9763e0a952b1f8f6e97c08d15d6d7096fc64a131d5527"}
Jul 30 22:48:16.157357 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.157348    2578 scope.go:117] "RemoveContainer" containerID="4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2"
Jul 30 22:48:16.157475 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.157297    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"
Jul 30 22:48:16.166279 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.166261    2578 scope.go:117] "RemoveContainer" containerID="4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2"
Jul 30 22:48:16.166542 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:16.166523    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2\": container with ID starting with 4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2 not found: ID does not exist" containerID="4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2"
Jul 30 22:48:16.166584 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.166550    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2"} err="failed to get container status \"4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2\": rpc error: code = NotFound desc = could not find container \"4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2\": container with ID starting with 4dbde09b2a8b851c2080e874bc735343518f562c8dd1411b879edce25534bbd2 not found: ID does not exist"
Jul 30 22:48:16.178612 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.178566    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7" podStartSLOduration=1.178551934 podStartE2EDuration="1.178551934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:16.177361854 +0000 UTC m=+476.352325527" watchObservedRunningTime="2026-07-30 22:48:16.178551934 +0000 UTC m=+476.353515613"
Jul 30 22:48:16.203182 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.203132    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"]
Jul 30 22:48:16.204814 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.204795    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-vrcmr"]
Jul 30 22:48:16.342118 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:16.342021    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="525e6b68-7b2b-4ad9-9a8a-526c63dbbac6" path="/var/lib/kubelet/pods/525e6b68-7b2b-4ad9-9a8a-526c63dbbac6/volumes"
Jul 30 22:48:27.165862 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:27.165824    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 22:48:27.224541 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:27.224500    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"]
Jul 30 22:48:27.224777 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:27.224753    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8" podUID="69cc1ef8-1338-416f-9ad8-938dddf96cdf" containerName="manager" containerID="cri-o://7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a" gracePeriod=10
Jul 30 22:48:27.964024 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:27.964000    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:28.146207 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.146119    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8prg\" (UniqueName: \"kubernetes.io/projected/69cc1ef8-1338-416f-9ad8-938dddf96cdf-kube-api-access-q8prg\") pod \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\" (UID: \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\") "
Jul 30 22:48:28.146207 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.146168    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69cc1ef8-1338-416f-9ad8-938dddf96cdf-extensions-socket-volume\") pod \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\" (UID: \"69cc1ef8-1338-416f-9ad8-938dddf96cdf\") "
Jul 30 22:48:28.146561 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.146540    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69cc1ef8-1338-416f-9ad8-938dddf96cdf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "69cc1ef8-1338-416f-9ad8-938dddf96cdf" (UID: "69cc1ef8-1338-416f-9ad8-938dddf96cdf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 22:48:28.148424 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.148403    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69cc1ef8-1338-416f-9ad8-938dddf96cdf-kube-api-access-q8prg" (OuterVolumeSpecName: "kube-api-access-q8prg") pod "69cc1ef8-1338-416f-9ad8-938dddf96cdf" (UID: "69cc1ef8-1338-416f-9ad8-938dddf96cdf"). InnerVolumeSpecName "kube-api-access-q8prg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:48:28.213196 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.213161    2578 generic.go:356] "Generic (PLEG): container finished" podID="69cc1ef8-1338-416f-9ad8-938dddf96cdf" containerID="7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a" exitCode=0
Jul 30 22:48:28.213648 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.213216    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"
Jul 30 22:48:28.213648 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.213244    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8" event={"ID":"69cc1ef8-1338-416f-9ad8-938dddf96cdf","Type":"ContainerDied","Data":"7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a"}
Jul 30 22:48:28.213648 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.213294    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8" event={"ID":"69cc1ef8-1338-416f-9ad8-938dddf96cdf","Type":"ContainerDied","Data":"76247f9e5902932fd4cf841823b6490a25b6ea4829328b991cfcf2d001ae9b9f"}
Jul 30 22:48:28.213648 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.213317    2578 scope.go:117] "RemoveContainer" containerID="7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a"
Jul 30 22:48:28.221978 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.221964    2578 scope.go:117] "RemoveContainer" containerID="7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a"
Jul 30 22:48:28.222268 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:28.222247    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a\": container with ID starting with 7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a not found: ID does not exist" containerID="7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a"
Jul 30 22:48:28.222347 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.222276    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a"} err="failed to get container status \"7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a\": rpc error: code = NotFound desc = could not find container \"7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a\": container with ID starting with 7504baa5f0c165e91396132c1f0d6af192e741ebd25350ea5ad4848056c3b69a not found: ID does not exist"
Jul 30 22:48:28.238459 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.238429    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"]
Jul 30 22:48:28.243280 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.243252    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-96db98c47-vtjs8"]
Jul 30 22:48:28.247119 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.247096    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8prg\" (UniqueName: \"kubernetes.io/projected/69cc1ef8-1338-416f-9ad8-938dddf96cdf-kube-api-access-q8prg\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:28.247119 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.247120    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69cc1ef8-1338-416f-9ad8-938dddf96cdf-extensions-socket-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:28.341483 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:28.341447    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69cc1ef8-1338-416f-9ad8-938dddf96cdf" path="/var/lib/kubelet/pods/69cc1ef8-1338-416f-9ad8-938dddf96cdf/volumes"
Jul 30 22:48:43.721769 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.721740    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"]
Jul 30 22:48:43.722273 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.722093    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69cc1ef8-1338-416f-9ad8-938dddf96cdf" containerName="manager"
Jul 30 22:48:43.722273 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.722106    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="69cc1ef8-1338-416f-9ad8-938dddf96cdf" containerName="manager"
Jul 30 22:48:43.722273 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.722175    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="69cc1ef8-1338-416f-9ad8-938dddf96cdf" containerName="manager"
Jul 30 22:48:43.725021 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.725003    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:43.728999 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.728976    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 30 22:48:43.734177 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.734153    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"]
Jul 30 22:48:43.768497 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.768463    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzdbk\" (UniqueName: \"kubernetes.io/projected/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-kube-api-access-lzdbk\") pod \"limitador-limitador-859c4d96d6-wmk9w\" (UID: \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:43.768649 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.768506    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-config-file\") pod \"limitador-limitador-859c4d96d6-wmk9w\" (UID: \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:43.826122 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.826083    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"]
Jul 30 22:48:43.869908 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.869870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzdbk\" (UniqueName: \"kubernetes.io/projected/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-kube-api-access-lzdbk\") pod \"limitador-limitador-859c4d96d6-wmk9w\" (UID: \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:43.870089 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.869918    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-config-file\") pod \"limitador-limitador-859c4d96d6-wmk9w\" (UID: \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:43.870547 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.870521    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-config-file\") pod \"limitador-limitador-859c4d96d6-wmk9w\" (UID: \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:43.878956 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:43.878937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzdbk\" (UniqueName: \"kubernetes.io/projected/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-kube-api-access-lzdbk\") pod \"limitador-limitador-859c4d96d6-wmk9w\" (UID: \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:44.035113 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.034986    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:44.178256 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.178216    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"]
Jul 30 22:48:44.181322 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:48:44.181293    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda62b1ab_1e9c_4f71_bbdc_113b66a3a516.slice/crio-0e1b3f829aa2578e5674d59786c878c775e232a1989375ef46900e409269d588 WatchSource:0}: Error finding container 0e1b3f829aa2578e5674d59786c878c775e232a1989375ef46900e409269d588: Status 404 returned error can't find the container with id 0e1b3f829aa2578e5674d59786c878c775e232a1989375ef46900e409269d588
Jul 30 22:48:44.283577 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.283542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w" event={"ID":"da62b1ab-1e9c-4f71-bbdc-113b66a3a516","Type":"ContainerStarted","Data":"0e1b3f829aa2578e5674d59786c878c775e232a1989375ef46900e409269d588"}
Jul 30 22:48:44.628707 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.628670    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-cdcws"]
Jul 30 22:48:44.632982 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.632961    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-cdcws"
Jul 30 22:48:44.635528 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.635511    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6lx46\""
Jul 30 22:48:44.642833 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.642810    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-cdcws"]
Jul 30 22:48:44.676542 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.676512    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9fmd\" (UniqueName: \"kubernetes.io/projected/ce7645a7-468c-4ad1-8be4-c79b26545e90-kube-api-access-w9fmd\") pod \"authorino-b54df94b4-cdcws\" (UID: \"ce7645a7-468c-4ad1-8be4-c79b26545e90\") " pod="kuadrant-system/authorino-b54df94b4-cdcws"
Jul 30 22:48:44.777165 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.777110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9fmd\" (UniqueName: \"kubernetes.io/projected/ce7645a7-468c-4ad1-8be4-c79b26545e90-kube-api-access-w9fmd\") pod \"authorino-b54df94b4-cdcws\" (UID: \"ce7645a7-468c-4ad1-8be4-c79b26545e90\") " pod="kuadrant-system/authorino-b54df94b4-cdcws"
Jul 30 22:48:44.792039 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.792004    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9fmd\" (UniqueName: \"kubernetes.io/projected/ce7645a7-468c-4ad1-8be4-c79b26545e90-kube-api-access-w9fmd\") pod \"authorino-b54df94b4-cdcws\" (UID: \"ce7645a7-468c-4ad1-8be4-c79b26545e90\") " pod="kuadrant-system/authorino-b54df94b4-cdcws"
Jul 30 22:48:44.943968 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:44.943937    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-cdcws"
Jul 30 22:48:45.079247 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:45.079213    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-cdcws"]
Jul 30 22:48:45.082344 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:48:45.082315    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce7645a7_468c_4ad1_8be4_c79b26545e90.slice/crio-e8af23e57626f7d1fc3eec69bdc6d9aeb51ed53a25de84c646cd1d3f13066e0e WatchSource:0}: Error finding container e8af23e57626f7d1fc3eec69bdc6d9aeb51ed53a25de84c646cd1d3f13066e0e: Status 404 returned error can't find the container with id e8af23e57626f7d1fc3eec69bdc6d9aeb51ed53a25de84c646cd1d3f13066e0e
Jul 30 22:48:45.290104 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:45.290003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-cdcws" event={"ID":"ce7645a7-468c-4ad1-8be4-c79b26545e90","Type":"ContainerStarted","Data":"e8af23e57626f7d1fc3eec69bdc6d9aeb51ed53a25de84c646cd1d3f13066e0e"}
Jul 30 22:48:46.299838 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:46.299798    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w" event={"ID":"da62b1ab-1e9c-4f71-bbdc-113b66a3a516","Type":"ContainerStarted","Data":"ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6"}
Jul 30 22:48:46.300359 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:46.300112    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:46.322077 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:46.321974    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w" podStartSLOduration=1.908423142 podStartE2EDuration="3.321941667s" totalImagesPullingTime="1.413518525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:46.318834438 +0000 UTC m=+506.493798118" watchObservedRunningTime="2026-07-30 22:48:46.321941667 +0000 UTC m=+506.496905346"
Jul 30 22:48:47.305334 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:47.305238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-cdcws" event={"ID":"ce7645a7-468c-4ad1-8be4-c79b26545e90","Type":"ContainerStarted","Data":"23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf"}
Jul 30 22:48:47.322046 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:47.321995    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-cdcws" podStartSLOduration=1.3760347419999999 podStartE2EDuration="3.321977436s" totalImagesPullingTime="1.945942694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:48:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:48:47.320701218 +0000 UTC m=+507.495664898" watchObservedRunningTime="2026-07-30 22:48:47.321977436 +0000 UTC m=+507.496941114"
Jul 30 22:48:48.107529 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:48.107485    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-cdcws"]
Jul 30 22:48:49.314987 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:49.314939    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-cdcws" podUID="ce7645a7-468c-4ad1-8be4-c79b26545e90" containerName="authorino" containerID="cri-o://23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf" gracePeriod=30
Jul 30 22:48:49.556376 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:49.556348    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-cdcws"
Jul 30 22:48:49.617764 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:49.617671    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9fmd\" (UniqueName: \"kubernetes.io/projected/ce7645a7-468c-4ad1-8be4-c79b26545e90-kube-api-access-w9fmd\") pod \"ce7645a7-468c-4ad1-8be4-c79b26545e90\" (UID: \"ce7645a7-468c-4ad1-8be4-c79b26545e90\") "
Jul 30 22:48:49.619865 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:49.619833    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce7645a7-468c-4ad1-8be4-c79b26545e90-kube-api-access-w9fmd" (OuterVolumeSpecName: "kube-api-access-w9fmd") pod "ce7645a7-468c-4ad1-8be4-c79b26545e90" (UID: "ce7645a7-468c-4ad1-8be4-c79b26545e90"). InnerVolumeSpecName "kube-api-access-w9fmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:48:49.718673 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:49.718636    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9fmd\" (UniqueName: \"kubernetes.io/projected/ce7645a7-468c-4ad1-8be4-c79b26545e90-kube-api-access-w9fmd\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:48:50.320039 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:50.320004    2578 generic.go:356] "Generic (PLEG): container finished" podID="ce7645a7-468c-4ad1-8be4-c79b26545e90" containerID="23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf" exitCode=0
Jul 30 22:48:50.320457 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:50.320065    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-cdcws"
Jul 30 22:48:50.320457 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:50.320081    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-cdcws" event={"ID":"ce7645a7-468c-4ad1-8be4-c79b26545e90","Type":"ContainerDied","Data":"23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf"}
Jul 30 22:48:50.320457 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:50.320124    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-cdcws" event={"ID":"ce7645a7-468c-4ad1-8be4-c79b26545e90","Type":"ContainerDied","Data":"e8af23e57626f7d1fc3eec69bdc6d9aeb51ed53a25de84c646cd1d3f13066e0e"}
Jul 30 22:48:50.320457 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:50.320140    2578 scope.go:117] "RemoveContainer" containerID="23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf"
Jul 30 22:48:50.329023 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:50.329002    2578 scope.go:117] "RemoveContainer" containerID="23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf"
Jul 30 22:48:50.329352 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:48:50.329331    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf\": container with ID starting with 23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf not found: ID does not exist" containerID="23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf"
Jul 30 22:48:50.329396 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:50.329362    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf"} err="failed to get container status \"23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf\": rpc error: code = NotFound desc = could not find container \"23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf\": container with ID starting with 23a2db4b73f859221ca09b9fac3a8145a1859d633487b6d9a43b7b90d42c0abf not found: ID does not exist"
Jul 30 22:48:50.342256 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:50.342231    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-cdcws"]
Jul 30 22:48:50.346662 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:50.346643    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-cdcws"]
Jul 30 22:48:52.345197 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:52.342422    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce7645a7-468c-4ad1-8be4-c79b26545e90" path="/var/lib/kubelet/pods/ce7645a7-468c-4ad1-8be4-c79b26545e90/volumes"
Jul 30 22:48:57.306727 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:57.306692    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:48:59.116217 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:59.116185    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"]
Jul 30 22:48:59.116671 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:48:59.116404    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w" podUID="da62b1ab-1e9c-4f71-bbdc-113b66a3a516" containerName="limitador" containerID="cri-o://ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6" gracePeriod=30
Jul 30 22:49:00.059182 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.059157    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:49:00.098186 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.098156    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-config-file\") pod \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\" (UID: \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\") "
Jul 30 22:49:00.098344 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.098202    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzdbk\" (UniqueName: \"kubernetes.io/projected/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-kube-api-access-lzdbk\") pod \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\" (UID: \"da62b1ab-1e9c-4f71-bbdc-113b66a3a516\") "
Jul 30 22:49:00.098592 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.098558    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-config-file" (OuterVolumeSpecName: "config-file") pod "da62b1ab-1e9c-4f71-bbdc-113b66a3a516" (UID: "da62b1ab-1e9c-4f71-bbdc-113b66a3a516"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 22:49:00.100452 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.100425    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-kube-api-access-lzdbk" (OuterVolumeSpecName: "kube-api-access-lzdbk") pod "da62b1ab-1e9c-4f71-bbdc-113b66a3a516" (UID: "da62b1ab-1e9c-4f71-bbdc-113b66a3a516"). InnerVolumeSpecName "kube-api-access-lzdbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:49:00.198903 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.198871    2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-config-file\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:49:00.198903 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.198905    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzdbk\" (UniqueName: \"kubernetes.io/projected/da62b1ab-1e9c-4f71-bbdc-113b66a3a516-kube-api-access-lzdbk\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:49:00.362758 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.362723    2578 generic.go:356] "Generic (PLEG): container finished" podID="da62b1ab-1e9c-4f71-bbdc-113b66a3a516" containerID="ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6" exitCode=0
Jul 30 22:49:00.362926 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.362761    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w" event={"ID":"da62b1ab-1e9c-4f71-bbdc-113b66a3a516","Type":"ContainerDied","Data":"ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6"}
Jul 30 22:49:00.362926 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.362813    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"
Jul 30 22:49:00.362926 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.362824    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-wmk9w" event={"ID":"da62b1ab-1e9c-4f71-bbdc-113b66a3a516","Type":"ContainerDied","Data":"0e1b3f829aa2578e5674d59786c878c775e232a1989375ef46900e409269d588"}
Jul 30 22:49:00.362926 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.362853    2578 scope.go:117] "RemoveContainer" containerID="ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6"
Jul 30 22:49:00.372391 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.372373    2578 scope.go:117] "RemoveContainer" containerID="ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6"
Jul 30 22:49:00.372640 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:49:00.372621    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6\": container with ID starting with ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6 not found: ID does not exist" containerID="ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6"
Jul 30 22:49:00.372703 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.372653    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6"} err="failed to get container status \"ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6\": rpc error: code = NotFound desc = could not find container \"ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6\": container with ID starting with ecda525e6ca98e3f338efe32bcbf438c6a0342e127868626859d3b56d00153e6 not found: ID does not exist"
Jul 30 22:49:00.380226 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.380204    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"]
Jul 30 22:49:00.384881 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:00.384858    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-wmk9w"]
Jul 30 22:49:01.018684 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.018654    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"]
Jul 30 22:49:01.018962 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.018951    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce7645a7-468c-4ad1-8be4-c79b26545e90" containerName="authorino"
Jul 30 22:49:01.019002 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.018965    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce7645a7-468c-4ad1-8be4-c79b26545e90" containerName="authorino"
Jul 30 22:49:01.019002 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.018979    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da62b1ab-1e9c-4f71-bbdc-113b66a3a516" containerName="limitador"
Jul 30 22:49:01.019002 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.018984    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="da62b1ab-1e9c-4f71-bbdc-113b66a3a516" containerName="limitador"
Jul 30 22:49:01.019113 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.019035    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="da62b1ab-1e9c-4f71-bbdc-113b66a3a516" containerName="limitador"
Jul 30 22:49:01.019113 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.019043    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce7645a7-468c-4ad1-8be4-c79b26545e90" containerName="authorino"
Jul 30 22:49:01.023254 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.023236    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:01.029505 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.029484    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-sj85l\""
Jul 30 22:49:01.030291 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.030275    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 30 22:49:01.030339 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.030307    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 30 22:49:01.030548 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.030535    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 30 22:49:01.040481 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.040456    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"]
Jul 30 22:49:01.106419 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.106378    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlsbx\" (UniqueName: \"kubernetes.io/projected/9dcdc4b6-ce5d-496f-af33-2a615f771ae4-kube-api-access-mlsbx\") pod \"postgres-69fb4594cf-skrkb\" (UID: \"9dcdc4b6-ce5d-496f-af33-2a615f771ae4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:01.106419 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.106420    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9dcdc4b6-ce5d-496f-af33-2a615f771ae4-data\") pod \"postgres-69fb4594cf-skrkb\" (UID: \"9dcdc4b6-ce5d-496f-af33-2a615f771ae4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:01.207251 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.207196    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlsbx\" (UniqueName: \"kubernetes.io/projected/9dcdc4b6-ce5d-496f-af33-2a615f771ae4-kube-api-access-mlsbx\") pod \"postgres-69fb4594cf-skrkb\" (UID: \"9dcdc4b6-ce5d-496f-af33-2a615f771ae4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:01.207251 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.207254    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9dcdc4b6-ce5d-496f-af33-2a615f771ae4-data\") pod \"postgres-69fb4594cf-skrkb\" (UID: \"9dcdc4b6-ce5d-496f-af33-2a615f771ae4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:01.207782 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.207701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/9dcdc4b6-ce5d-496f-af33-2a615f771ae4-data\") pod \"postgres-69fb4594cf-skrkb\" (UID: \"9dcdc4b6-ce5d-496f-af33-2a615f771ae4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:01.218148 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.218116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlsbx\" (UniqueName: \"kubernetes.io/projected/9dcdc4b6-ce5d-496f-af33-2a615f771ae4-kube-api-access-mlsbx\") pod \"postgres-69fb4594cf-skrkb\" (UID: \"9dcdc4b6-ce5d-496f-af33-2a615f771ae4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:01.333000 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.332909    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:01.482406 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:01.482249    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"]
Jul 30 22:49:01.485031 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:49:01.485004    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dcdc4b6_ce5d_496f_af33_2a615f771ae4.slice/crio-296d29c9958c206538941519d66d74440a0e9251718181218bdb90d5c660e971 WatchSource:0}: Error finding container 296d29c9958c206538941519d66d74440a0e9251718181218bdb90d5c660e971: Status 404 returned error can't find the container with id 296d29c9958c206538941519d66d74440a0e9251718181218bdb90d5c660e971
Jul 30 22:49:02.343890 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:02.343843    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da62b1ab-1e9c-4f71-bbdc-113b66a3a516" path="/var/lib/kubelet/pods/da62b1ab-1e9c-4f71-bbdc-113b66a3a516/volumes"
Jul 30 22:49:02.376290 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:02.376255    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb" event={"ID":"9dcdc4b6-ce5d-496f-af33-2a615f771ae4","Type":"ContainerStarted","Data":"296d29c9958c206538941519d66d74440a0e9251718181218bdb90d5c660e971"}
Jul 30 22:49:07.401963 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:07.401923    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb" event={"ID":"9dcdc4b6-ce5d-496f-af33-2a615f771ae4","Type":"ContainerStarted","Data":"f9976065e180daf13f845b14087696b91d8ac96a2db9f0f4271553f58efb159e"}
Jul 30 22:49:07.402343 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:07.402069    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:07.420804 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:07.420724    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb" podStartSLOduration=2.5161896649999997 podStartE2EDuration="7.420707033s" totalImagesPullingTime="4.904517368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:49:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:49:07.419658488 +0000 UTC m=+527.594622167" watchObservedRunningTime="2026-07-30 22:49:07.420707033 +0000 UTC m=+527.595670711"
Jul 30 22:49:13.437365 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:13.437282    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-skrkb"
Jul 30 22:49:14.010858 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.010822    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-prxnx"]
Jul 30 22:49:14.084646 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.084608    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-prxnx"]
Jul 30 22:49:14.084812 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.084735    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-prxnx"
Jul 30 22:49:14.089029 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.089006    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6lx46\""
Jul 30 22:49:14.159535 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.159499    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-prxnx"]
Jul 30 22:49:14.159752 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:49:14.159731    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-chrp8], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5dc7ccb5f-prxnx" podUID="85128235-6f1b-4eae-9fd1-f3e275188ef4"
Jul 30 22:49:14.216203 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.216174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chrp8\" (UniqueName: \"kubernetes.io/projected/85128235-6f1b-4eae-9fd1-f3e275188ef4-kube-api-access-chrp8\") pod \"authorino-5dc7ccb5f-prxnx\" (UID: \"85128235-6f1b-4eae-9fd1-f3e275188ef4\") " pod="kuadrant-system/authorino-5dc7ccb5f-prxnx"
Jul 30 22:49:14.313427 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.313348    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55ffdf754d-2p7jd"]
Jul 30 22:49:14.317141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.317115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chrp8\" (UniqueName: \"kubernetes.io/projected/85128235-6f1b-4eae-9fd1-f3e275188ef4-kube-api-access-chrp8\") pod \"authorino-5dc7ccb5f-prxnx\" (UID: \"85128235-6f1b-4eae-9fd1-f3e275188ef4\") " pod="kuadrant-system/authorino-5dc7ccb5f-prxnx"
Jul 30 22:49:14.332230 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.332197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chrp8\" (UniqueName: \"kubernetes.io/projected/85128235-6f1b-4eae-9fd1-f3e275188ef4-kube-api-access-chrp8\") pod \"authorino-5dc7ccb5f-prxnx\" (UID: \"85128235-6f1b-4eae-9fd1-f3e275188ef4\") " pod="kuadrant-system/authorino-5dc7ccb5f-prxnx"
Jul 30 22:49:14.342581 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.342549    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:49:14.345552 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.345530    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 30 22:49:14.348088 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.348020    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55ffdf754d-2p7jd"]
Jul 30 22:49:14.417578 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.417543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c30db949-186e-4f18-a4a4-38e95cfb41e0-tls-cert\") pod \"authorino-55ffdf754d-2p7jd\" (UID: \"c30db949-186e-4f18-a4a4-38e95cfb41e0\") " pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:49:14.417773 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.417606    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmkl5\" (UniqueName: \"kubernetes.io/projected/c30db949-186e-4f18-a4a4-38e95cfb41e0-kube-api-access-tmkl5\") pod \"authorino-55ffdf754d-2p7jd\" (UID: \"c30db949-186e-4f18-a4a4-38e95cfb41e0\") " pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:49:14.436172 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.436138    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-prxnx"
Jul 30 22:49:14.483392 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.483359    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-prxnx"
Jul 30 22:49:14.518313 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.518280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c30db949-186e-4f18-a4a4-38e95cfb41e0-tls-cert\") pod \"authorino-55ffdf754d-2p7jd\" (UID: \"c30db949-186e-4f18-a4a4-38e95cfb41e0\") " pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:49:14.518459 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.518336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmkl5\" (UniqueName: \"kubernetes.io/projected/c30db949-186e-4f18-a4a4-38e95cfb41e0-kube-api-access-tmkl5\") pod \"authorino-55ffdf754d-2p7jd\" (UID: \"c30db949-186e-4f18-a4a4-38e95cfb41e0\") " pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:49:14.520987 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.520964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c30db949-186e-4f18-a4a4-38e95cfb41e0-tls-cert\") pod \"authorino-55ffdf754d-2p7jd\" (UID: \"c30db949-186e-4f18-a4a4-38e95cfb41e0\") " pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:49:14.527921 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.527888    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmkl5\" (UniqueName: \"kubernetes.io/projected/c30db949-186e-4f18-a4a4-38e95cfb41e0-kube-api-access-tmkl5\") pod \"authorino-55ffdf754d-2p7jd\" (UID: \"c30db949-186e-4f18-a4a4-38e95cfb41e0\") " pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:49:14.619035 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.618955    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chrp8\" (UniqueName: \"kubernetes.io/projected/85128235-6f1b-4eae-9fd1-f3e275188ef4-kube-api-access-chrp8\") pod \"85128235-6f1b-4eae-9fd1-f3e275188ef4\" (UID: \"85128235-6f1b-4eae-9fd1-f3e275188ef4\") "
Jul 30 22:49:14.621304 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.621271    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85128235-6f1b-4eae-9fd1-f3e275188ef4-kube-api-access-chrp8" (OuterVolumeSpecName: "kube-api-access-chrp8") pod "85128235-6f1b-4eae-9fd1-f3e275188ef4" (UID: "85128235-6f1b-4eae-9fd1-f3e275188ef4"). InnerVolumeSpecName "kube-api-access-chrp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:49:14.652584 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.652554    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:49:14.720256 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.720219    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chrp8\" (UniqueName: \"kubernetes.io/projected/85128235-6f1b-4eae-9fd1-f3e275188ef4-kube-api-access-chrp8\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:49:14.825963 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:14.825932    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55ffdf754d-2p7jd"]
Jul 30 22:49:14.828984 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:49:14.828955    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc30db949_186e_4f18_a4a4_38e95cfb41e0.slice/crio-6d509108518a328223a065882fc1109e718a8eb6703ee78fc28e19de268aca84 WatchSource:0}: Error finding container 6d509108518a328223a065882fc1109e718a8eb6703ee78fc28e19de268aca84: Status 404 returned error can't find the container with id 6d509108518a328223a065882fc1109e718a8eb6703ee78fc28e19de268aca84
Jul 30 22:49:15.442163 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:15.442125    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55ffdf754d-2p7jd" event={"ID":"c30db949-186e-4f18-a4a4-38e95cfb41e0","Type":"ContainerStarted","Data":"6d509108518a328223a065882fc1109e718a8eb6703ee78fc28e19de268aca84"}
Jul 30 22:49:15.442163 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:15.442149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5dc7ccb5f-prxnx"
Jul 30 22:49:15.473452 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:15.473413    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-prxnx"]
Jul 30 22:49:15.477750 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:15.477726    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5dc7ccb5f-prxnx"]
Jul 30 22:49:16.341599 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:16.341566    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85128235-6f1b-4eae-9fd1-f3e275188ef4" path="/var/lib/kubelet/pods/85128235-6f1b-4eae-9fd1-f3e275188ef4/volumes"
Jul 30 22:49:16.449311 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:16.449274    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55ffdf754d-2p7jd" event={"ID":"c30db949-186e-4f18-a4a4-38e95cfb41e0","Type":"ContainerStarted","Data":"994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de"}
Jul 30 22:49:16.475424 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:16.475374    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-55ffdf754d-2p7jd" podStartSLOduration=1.652873134 podStartE2EDuration="2.475360268s" totalImagesPullingTime="822.487134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:49:16.475273954 +0000 UTC m=+536.650237629" watchObservedRunningTime="2026-07-30 22:49:16.475360268 +0000 UTC m=+536.650323946"
Jul 30 22:49:57.002878 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.002843    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-n2d2v"]
Jul 30 22:49:57.006199 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.006181    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:49:57.009610 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.009583    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 30 22:49:57.009743 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.009644    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8dwqv\""
Jul 30 22:49:57.018389 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.018360    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-n2d2v"]
Jul 30 22:49:57.065448 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.065416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2j9q\" (UniqueName: \"kubernetes.io/projected/b9801adf-19b8-4d27-b493-69577a9e4c33-kube-api-access-b2j9q\") pod \"payload-processing-5cc6cd444b-n2d2v\" (UID: \"b9801adf-19b8-4d27-b493-69577a9e4c33\") " pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:49:57.065448 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.065452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9801adf-19b8-4d27-b493-69577a9e4c33-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-n2d2v\" (UID: \"b9801adf-19b8-4d27-b493-69577a9e4c33\") " pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:49:57.166335 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.166302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2j9q\" (UniqueName: \"kubernetes.io/projected/b9801adf-19b8-4d27-b493-69577a9e4c33-kube-api-access-b2j9q\") pod \"payload-processing-5cc6cd444b-n2d2v\" (UID: \"b9801adf-19b8-4d27-b493-69577a9e4c33\") " pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:49:57.166500 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.166345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9801adf-19b8-4d27-b493-69577a9e4c33-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-n2d2v\" (UID: \"b9801adf-19b8-4d27-b493-69577a9e4c33\") " pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:49:57.167087 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.167038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9801adf-19b8-4d27-b493-69577a9e4c33-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-n2d2v\" (UID: \"b9801adf-19b8-4d27-b493-69577a9e4c33\") " pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:49:57.181314 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.181278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2j9q\" (UniqueName: \"kubernetes.io/projected/b9801adf-19b8-4d27-b493-69577a9e4c33-kube-api-access-b2j9q\") pod \"payload-processing-5cc6cd444b-n2d2v\" (UID: \"b9801adf-19b8-4d27-b493-69577a9e4c33\") " pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:49:57.316382 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.316288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:49:57.451404 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.451380    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-n2d2v"]
Jul 30 22:49:57.454355 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:49:57.454328    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9801adf_19b8_4d27_b493_69577a9e4c33.slice/crio-b695e5cadac38c3efe21cc892c8bea45ce752dcb91b4fcb55c30067c96991553 WatchSource:0}: Error finding container b695e5cadac38c3efe21cc892c8bea45ce752dcb91b4fcb55c30067c96991553: Status 404 returned error can't find the container with id b695e5cadac38c3efe21cc892c8bea45ce752dcb91b4fcb55c30067c96991553
Jul 30 22:49:57.622462 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:49:57.622378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v" event={"ID":"b9801adf-19b8-4d27-b493-69577a9e4c33","Type":"ContainerStarted","Data":"b695e5cadac38c3efe21cc892c8bea45ce752dcb91b4fcb55c30067c96991553"}
Jul 30 22:50:00.636803 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:00.636768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v" event={"ID":"b9801adf-19b8-4d27-b493-69577a9e4c33","Type":"ContainerStarted","Data":"a97b6f9bd23e5809abaa7cdbb9d3375c1084077ab6edc7d439f60e0e163e2c49"}
Jul 30 22:50:00.637177 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:00.636902    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:50:00.653414 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:00.653358    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v" podStartSLOduration=2.404953493 podStartE2EDuration="4.653340075s" totalImagesPullingTime="2.248386582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:50:00.65303321 +0000 UTC m=+580.827996891" watchObservedRunningTime="2026-07-30 22:50:00.653340075 +0000 UTC m=+580.828303783"
Jul 30 22:50:11.642867 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:11.642833    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-n2d2v"
Jul 30 22:50:20.223229 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:20.223204    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:50:20.223229 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:20.223216    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:50:30.951827 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.951791    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"]
Jul 30 22:50:30.955477 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.955454    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:30.959888 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.959861    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 30 22:50:30.959888 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.959884    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lvhf4\""
Jul 30 22:50:30.960112 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.959873    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 30 22:50:30.960112 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.960017    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 30 22:50:30.962302 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.961936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:30.962302 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.962002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:30.962302 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.962035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pffsb\" (UniqueName: \"kubernetes.io/projected/e086a0bb-93bd-4b2c-979b-c7c77628ae74-kube-api-access-pffsb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:30.962302 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.962109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e086a0bb-93bd-4b2c-979b-c7c77628ae74-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:30.962302 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.962140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:30.962302 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.962189    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:30.968759 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:30.968731    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"]
Jul 30 22:50:31.063190 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.063150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.063388 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.063204    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.063388 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.063234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pffsb\" (UniqueName: \"kubernetes.io/projected/e086a0bb-93bd-4b2c-979b-c7c77628ae74-kube-api-access-pffsb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.063388 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.063269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e086a0bb-93bd-4b2c-979b-c7c77628ae74-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.063388 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.063296    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.063388 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.063373    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.063674 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.063654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.063739 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.063691    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.063739 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.063726    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.065571 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.065548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e086a0bb-93bd-4b2c-979b-c7c77628ae74-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.065934 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.065911    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e086a0bb-93bd-4b2c-979b-c7c77628ae74-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.076265 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.076235    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pffsb\" (UniqueName: \"kubernetes.io/projected/e086a0bb-93bd-4b2c-979b-c7c77628ae74-kube-api-access-pffsb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6grqg\" (UID: \"e086a0bb-93bd-4b2c-979b-c7c77628ae74\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.265837 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.265762    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:31.404854 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.404824    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"]
Jul 30 22:50:31.408316 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:50:31.408291    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode086a0bb_93bd_4b2c_979b_c7c77628ae74.slice/crio-3f127b5cda9ddc3aa7cdde33bea9bd4c12256f51470bec096b5833b8029e6528 WatchSource:0}: Error finding container 3f127b5cda9ddc3aa7cdde33bea9bd4c12256f51470bec096b5833b8029e6528: Status 404 returned error can't find the container with id 3f127b5cda9ddc3aa7cdde33bea9bd4c12256f51470bec096b5833b8029e6528
Jul 30 22:50:31.781265 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:31.781233    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg" event={"ID":"e086a0bb-93bd-4b2c-979b-c7c77628ae74","Type":"ContainerStarted","Data":"3f127b5cda9ddc3aa7cdde33bea9bd4c12256f51470bec096b5833b8029e6528"}
Jul 30 22:50:32.787224 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:32.787186    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg" event={"ID":"e086a0bb-93bd-4b2c-979b-c7c77628ae74","Type":"ContainerStarted","Data":"e8c64a6c7890253ff06fdb938c3c3ee8f93e4ea17f0b75de169843fff5b104f7"}
Jul 30 22:50:32.806582 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:32.806479    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg" podStartSLOduration=1.6738274149999999 podStartE2EDuration="2.806459629s" totalImagesPullingTime="1.132632214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:50:32.805477887 +0000 UTC m=+612.980441778" watchObservedRunningTime="2026-07-30 22:50:32.806459629 +0000 UTC m=+612.981423308"
Jul 30 22:50:38.397467 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.397433    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"]
Jul 30 22:50:38.399764 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.399743    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.402609 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.402590    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 30 22:50:38.416014 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.415984    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"]
Jul 30 22:50:38.425002 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.424976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f08a97-68b6-49c0-b12e-59dcb6067222-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.425185 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.425012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w779\" (UniqueName: \"kubernetes.io/projected/33f08a97-68b6-49c0-b12e-59dcb6067222-kube-api-access-5w779\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.425185 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.425068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.425185 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.425097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.425185 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.425119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.425185 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.425140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.525933 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.525902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w779\" (UniqueName: \"kubernetes.io/projected/33f08a97-68b6-49c0-b12e-59dcb6067222-kube-api-access-5w779\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.526125 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.525950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.526125 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.525978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.526125 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.526003    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.526125 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.526032    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.526125 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.526116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f08a97-68b6-49c0-b12e-59dcb6067222-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.526521 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.526491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.526601 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.526500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.526601 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.526567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.528368 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.528349    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33f08a97-68b6-49c0-b12e-59dcb6067222-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.528673 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.528655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33f08a97-68b6-49c0-b12e-59dcb6067222-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.535711 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.535681    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w779\" (UniqueName: \"kubernetes.io/projected/33f08a97-68b6-49c0-b12e-59dcb6067222-kube-api-access-5w779\") pod \"e2e-trlp-test-simulated-kserve-598466548d-p9wr2\" (UID: \"33f08a97-68b6-49c0-b12e-59dcb6067222\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.709141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.709098    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:38.850642 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.850613    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"]
Jul 30 22:50:38.853109 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:50:38.853079    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33f08a97_68b6_49c0_b12e_59dcb6067222.slice/crio-3e8f6d70af27c5d18c2cb725977c6126b0585172c857c330a74ac164161c4bb9 WatchSource:0}: Error finding container 3e8f6d70af27c5d18c2cb725977c6126b0585172c857c330a74ac164161c4bb9: Status 404 returned error can't find the container with id 3e8f6d70af27c5d18c2cb725977c6126b0585172c857c330a74ac164161c4bb9
Jul 30 22:50:38.854427 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:38.854411    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 22:50:39.821018 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:39.820973    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2" event={"ID":"33f08a97-68b6-49c0-b12e-59dcb6067222","Type":"ContainerStarted","Data":"c7af58e3a7ac4252d9afe3d57ca74e98e40c66f8975e0c117240a1c897771c39"}
Jul 30 22:50:39.821503 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:39.821068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2" event={"ID":"33f08a97-68b6-49c0-b12e-59dcb6067222","Type":"ContainerStarted","Data":"3e8f6d70af27c5d18c2cb725977c6126b0585172c857c330a74ac164161c4bb9"}
Jul 30 22:50:39.843072 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:39.842991    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2" podStartSLOduration=1.66485553 podStartE2EDuration="1.84297053s" totalImagesPullingTime="178.115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:50:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:50:39.842826392 +0000 UTC m=+620.017790072" watchObservedRunningTime="2026-07-30 22:50:39.84297053 +0000 UTC m=+620.017934211"
Jul 30 22:50:41.266037 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:41.265951    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:41.266037 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:41.266023    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:41.278553 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:41.278523    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:41.840846 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:41.840819    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6grqg"
Jul 30 22:50:48.709867 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:48.709820    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:48.710335 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:48.709903    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:48.722726 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:48.722693    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:50:48.872319 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:50:48.872289    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-p9wr2"
Jul 30 22:51:14.902937 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:14.902900    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"]
Jul 30 22:51:14.905588 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:14.905565    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:14.908806 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:14.908784    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 30 22:51:14.918560 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:14.918533    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"]
Jul 30 22:51:15.046508 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.046459    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.046704 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.046547    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.046704 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.046626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.046704 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.046682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.046862 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.046744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.046862 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.046770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p6sn\" (UniqueName: \"kubernetes.io/projected/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-kube-api-access-8p6sn\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.148076 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.148025    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.148076 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.148079    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.148342 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.148108    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.148342 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.148137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.148342 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.148164    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p6sn\" (UniqueName: \"kubernetes.io/projected/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-kube-api-access-8p6sn\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.148342 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.148215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.148583 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.148518    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.148583 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.148536    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.148684 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.148585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.150596 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.150557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.150931 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.150908    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.156961 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.156896    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p6sn\" (UniqueName: \"kubernetes.io/projected/fec0f14c-a1a0-4b3c-95de-24ddb8efb612-kube-api-access-8p6sn\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk\" (UID: \"fec0f14c-a1a0-4b3c-95de-24ddb8efb612\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.215819 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.215780    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:15.352654 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.352622    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"]
Jul 30 22:51:15.355851 ip-10-0-142-225 kubenswrapper[2578]: W0730 22:51:15.355820    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfec0f14c_a1a0_4b3c_95de_24ddb8efb612.slice/crio-a9e972969bff767f437a9e11915a53f90e55ff9204856be83db02701de0e4e41 WatchSource:0}: Error finding container a9e972969bff767f437a9e11915a53f90e55ff9204856be83db02701de0e4e41: Status 404 returned error can't find the container with id a9e972969bff767f437a9e11915a53f90e55ff9204856be83db02701de0e4e41
Jul 30 22:51:15.981186 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.981150    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk" event={"ID":"fec0f14c-a1a0-4b3c-95de-24ddb8efb612","Type":"ContainerStarted","Data":"7078610fdd718ce5e5ddfc04c2a3da830c31a9eabe8bd77863cb215169275c2c"}
Jul 30 22:51:15.981579 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.981202    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk" event={"ID":"fec0f14c-a1a0-4b3c-95de-24ddb8efb612","Type":"ContainerStarted","Data":"a9e972969bff767f437a9e11915a53f90e55ff9204856be83db02701de0e4e41"}
Jul 30 22:51:15.999621 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:15.999574    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk" podStartSLOduration=1.7789402760000002 podStartE2EDuration="1.999558195s" totalImagesPullingTime="220.617919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 22:51:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 22:51:15.998926138 +0000 UTC m=+656.173889817" watchObservedRunningTime="2026-07-30 22:51:15.999558195 +0000 UTC m=+656.174521895"
Jul 30 22:51:25.216070 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:25.216023    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:25.216551 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:25.216088    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:25.228757 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:25.228733    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:51:26.041319 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:51:26.041289    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk"
Jul 30 22:52:14.181124 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:14.180426    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55ffdf754d-2p7jd"]
Jul 30 22:52:14.181124 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:14.180687    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-55ffdf754d-2p7jd" podUID="c30db949-186e-4f18-a4a4-38e95cfb41e0" containerName="authorino" containerID="cri-o://994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de" gracePeriod=30
Jul 30 22:52:14.440904 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:14.440878    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:52:14.458251 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:14.458218    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c30db949-186e-4f18-a4a4-38e95cfb41e0-tls-cert\") pod \"c30db949-186e-4f18-a4a4-38e95cfb41e0\" (UID: \"c30db949-186e-4f18-a4a4-38e95cfb41e0\") "
Jul 30 22:52:14.458434 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:14.458299    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmkl5\" (UniqueName: \"kubernetes.io/projected/c30db949-186e-4f18-a4a4-38e95cfb41e0-kube-api-access-tmkl5\") pod \"c30db949-186e-4f18-a4a4-38e95cfb41e0\" (UID: \"c30db949-186e-4f18-a4a4-38e95cfb41e0\") "
Jul 30 22:52:14.461173 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:14.461134    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30db949-186e-4f18-a4a4-38e95cfb41e0-kube-api-access-tmkl5" (OuterVolumeSpecName: "kube-api-access-tmkl5") pod "c30db949-186e-4f18-a4a4-38e95cfb41e0" (UID: "c30db949-186e-4f18-a4a4-38e95cfb41e0"). InnerVolumeSpecName "kube-api-access-tmkl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 22:52:14.470202 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:14.470170    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c30db949-186e-4f18-a4a4-38e95cfb41e0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c30db949-186e-4f18-a4a4-38e95cfb41e0" (UID: "c30db949-186e-4f18-a4a4-38e95cfb41e0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 22:52:14.559631 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:14.559592    2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c30db949-186e-4f18-a4a4-38e95cfb41e0-tls-cert\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:52:14.559631 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:14.559623    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmkl5\" (UniqueName: \"kubernetes.io/projected/c30db949-186e-4f18-a4a4-38e95cfb41e0-kube-api-access-tmkl5\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 22:52:15.251897 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:15.251859    2578 generic.go:356] "Generic (PLEG): container finished" podID="c30db949-186e-4f18-a4a4-38e95cfb41e0" containerID="994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de" exitCode=0
Jul 30 22:52:15.252451 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:15.251910    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55ffdf754d-2p7jd"
Jul 30 22:52:15.252451 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:15.251941    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55ffdf754d-2p7jd" event={"ID":"c30db949-186e-4f18-a4a4-38e95cfb41e0","Type":"ContainerDied","Data":"994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de"}
Jul 30 22:52:15.252451 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:15.251984    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-55ffdf754d-2p7jd" event={"ID":"c30db949-186e-4f18-a4a4-38e95cfb41e0","Type":"ContainerDied","Data":"6d509108518a328223a065882fc1109e718a8eb6703ee78fc28e19de268aca84"}
Jul 30 22:52:15.252451 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:15.252005    2578 scope.go:117] "RemoveContainer" containerID="994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de"
Jul 30 22:52:15.261138 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:15.261114    2578 scope.go:117] "RemoveContainer" containerID="994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de"
Jul 30 22:52:15.261465 ip-10-0-142-225 kubenswrapper[2578]: E0730 22:52:15.261444    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de\": container with ID starting with 994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de not found: ID does not exist" containerID="994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de"
Jul 30 22:52:15.261524 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:15.261475    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de"} err="failed to get container status \"994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de\": rpc error: code = NotFound desc = could not find container \"994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de\": container with ID starting with 994e60ce211b36b6e22a43e81e86a82d9ac509d9b9c977160d2b7aba7d4283de not found: ID does not exist"
Jul 30 22:52:15.280141 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:15.280107    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55ffdf754d-2p7jd"]
Jul 30 22:52:15.284892 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:15.284855    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-55ffdf754d-2p7jd"]
Jul 30 22:52:16.342492 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:52:16.342449    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30db949-186e-4f18-a4a4-38e95cfb41e0" path="/var/lib/kubelet/pods/c30db949-186e-4f18-a4a4-38e95cfb41e0/volumes"
Jul 30 22:55:20.254919 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:55:20.254888    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 22:55:20.255568 ip-10-0-142-225 kubenswrapper[2578]: I0730 22:55:20.255550    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:00:20.283288 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:20.283259    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:00:20.284238 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:20.284218    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:00:50.119920 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.119869    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"]
Jul 30 23:00:50.120502 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.120134    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7" podUID="12ec5f7e-acea-4304-a24c-4ec967075fc9" containerName="manager" containerID="cri-o://23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e" gracePeriod=10
Jul 30 23:00:50.371117 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.371043    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 23:00:50.427797 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.427768    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcj4m\" (UniqueName: \"kubernetes.io/projected/12ec5f7e-acea-4304-a24c-4ec967075fc9-kube-api-access-mcj4m\") pod \"12ec5f7e-acea-4304-a24c-4ec967075fc9\" (UID: \"12ec5f7e-acea-4304-a24c-4ec967075fc9\") "
Jul 30 23:00:50.427943 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.427870    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ec5f7e-acea-4304-a24c-4ec967075fc9-extensions-socket-volume\") pod \"12ec5f7e-acea-4304-a24c-4ec967075fc9\" (UID: \"12ec5f7e-acea-4304-a24c-4ec967075fc9\") "
Jul 30 23:00:50.428286 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.428250    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ec5f7e-acea-4304-a24c-4ec967075fc9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "12ec5f7e-acea-4304-a24c-4ec967075fc9" (UID: "12ec5f7e-acea-4304-a24c-4ec967075fc9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:00:50.429946 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.429923    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ec5f7e-acea-4304-a24c-4ec967075fc9-kube-api-access-mcj4m" (OuterVolumeSpecName: "kube-api-access-mcj4m") pod "12ec5f7e-acea-4304-a24c-4ec967075fc9" (UID: "12ec5f7e-acea-4304-a24c-4ec967075fc9"). InnerVolumeSpecName "kube-api-access-mcj4m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:00:50.529084 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.529024    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ec5f7e-acea-4304-a24c-4ec967075fc9-extensions-socket-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:00:50.530092 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.529305    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcj4m\" (UniqueName: \"kubernetes.io/projected/12ec5f7e-acea-4304-a24c-4ec967075fc9-kube-api-access-mcj4m\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:00:50.565306 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.565258    2578 generic.go:356] "Generic (PLEG): container finished" podID="12ec5f7e-acea-4304-a24c-4ec967075fc9" containerID="23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e" exitCode=0
Jul 30 23:00:50.565306 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.565300    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7" event={"ID":"12ec5f7e-acea-4304-a24c-4ec967075fc9","Type":"ContainerDied","Data":"23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e"}
Jul 30 23:00:50.565504 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.565316    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"
Jul 30 23:00:50.565504 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.565375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7" event={"ID":"12ec5f7e-acea-4304-a24c-4ec967075fc9","Type":"ContainerDied","Data":"8bec6177bd601e54698c3a0c83f4fa181818e044d9ba5cdc88679cab32f7aa19"}
Jul 30 23:00:50.565504 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.565397    2578 scope.go:117] "RemoveContainer" containerID="23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e"
Jul 30 23:00:50.575549 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.575528    2578 scope.go:117] "RemoveContainer" containerID="23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e"
Jul 30 23:00:50.575812 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:00:50.575788    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e\": container with ID starting with 23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e not found: ID does not exist" containerID="23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e"
Jul 30 23:00:50.575851 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.575825    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e"} err="failed to get container status \"23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e\": rpc error: code = NotFound desc = could not find container \"23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e\": container with ID starting with 23805b0d687931107a2cf2112731a5af249d7aa0d2f4c19093dc0b8cd6a4786e not found: ID does not exist"
Jul 30 23:00:50.586399 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.586375    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"]
Jul 30 23:00:50.591718 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:50.591693    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-jwnr7"]
Jul 30 23:00:52.342494 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:00:52.342460    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12ec5f7e-acea-4304-a24c-4ec967075fc9" path="/var/lib/kubelet/pods/12ec5f7e-acea-4304-a24c-4ec967075fc9/volumes"
Jul 30 23:01:50.842022 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.841990    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"]
Jul 30 23:01:50.844360 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.842319    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30db949-186e-4f18-a4a4-38e95cfb41e0" containerName="authorino"
Jul 30 23:01:50.844360 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.842330    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30db949-186e-4f18-a4a4-38e95cfb41e0" containerName="authorino"
Jul 30 23:01:50.844360 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.842344    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12ec5f7e-acea-4304-a24c-4ec967075fc9" containerName="manager"
Jul 30 23:01:50.844360 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.842350    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="12ec5f7e-acea-4304-a24c-4ec967075fc9" containerName="manager"
Jul 30 23:01:50.844360 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.842398    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="12ec5f7e-acea-4304-a24c-4ec967075fc9" containerName="manager"
Jul 30 23:01:50.844360 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.842407    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30db949-186e-4f18-a4a4-38e95cfb41e0" containerName="authorino"
Jul 30 23:01:50.845272 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.845247    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:01:50.847865 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.847848    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9q7ms\""
Jul 30 23:01:50.861112 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.861082    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"]
Jul 30 23:01:50.941628 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.941588    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9xcl\" (UniqueName: \"kubernetes.io/projected/e7d3df26-0872-4ac9-80fb-fc714bd557ee-kube-api-access-s9xcl\") pod \"kuadrant-operator-controller-manager-68cf589589-pgvc6\" (UID: \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:01:50.941810 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:50.941694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d3df26-0872-4ac9-80fb-fc714bd557ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-pgvc6\" (UID: \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:01:51.042193 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.042158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9xcl\" (UniqueName: \"kubernetes.io/projected/e7d3df26-0872-4ac9-80fb-fc714bd557ee-kube-api-access-s9xcl\") pod \"kuadrant-operator-controller-manager-68cf589589-pgvc6\" (UID: \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:01:51.042368 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.042231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d3df26-0872-4ac9-80fb-fc714bd557ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-pgvc6\" (UID: \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:01:51.042662 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.042643    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d3df26-0872-4ac9-80fb-fc714bd557ee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-pgvc6\" (UID: \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:01:51.056270 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.056245    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9xcl\" (UniqueName: \"kubernetes.io/projected/e7d3df26-0872-4ac9-80fb-fc714bd557ee-kube-api-access-s9xcl\") pod \"kuadrant-operator-controller-manager-68cf589589-pgvc6\" (UID: \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:01:51.155564 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.155473    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:01:51.297863 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.297837    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"]
Jul 30 23:01:51.300468 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:01:51.300443    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d3df26_0872_4ac9_80fb_fc714bd557ee.slice/crio-9bc1be4daed2d794475cd567a46bc7e7dc1dc6c81ece785bd85026147f6cf896 WatchSource:0}: Error finding container 9bc1be4daed2d794475cd567a46bc7e7dc1dc6c81ece785bd85026147f6cf896: Status 404 returned error can't find the container with id 9bc1be4daed2d794475cd567a46bc7e7dc1dc6c81ece785bd85026147f6cf896
Jul 30 23:01:51.831827 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.831791    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6" event={"ID":"e7d3df26-0872-4ac9-80fb-fc714bd557ee","Type":"ContainerStarted","Data":"f7005e9638b4b403250188a7db3ed32aa233433af4d716dce461d4db6df90b22"}
Jul 30 23:01:51.832005 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.831848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6" event={"ID":"e7d3df26-0872-4ac9-80fb-fc714bd557ee","Type":"ContainerStarted","Data":"9bc1be4daed2d794475cd567a46bc7e7dc1dc6c81ece785bd85026147f6cf896"}
Jul 30 23:01:51.832005 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.831904    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:01:51.856145 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:01:51.856086    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6" podStartSLOduration=1.856044028 podStartE2EDuration="1.856044028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:01:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:01:51.85426813 +0000 UTC m=+1292.029231809" watchObservedRunningTime="2026-07-30 23:01:51.856044028 +0000 UTC m=+1292.031007707"
Jul 30 23:02:02.838789 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:02:02.838756    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:04:42.702431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.702400    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"]
Jul 30 23:04:42.711363 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.711331    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.714317 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.714287    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-59ba5ab4-kserve-self-signed-certs\""
Jul 30 23:04:42.723625 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.723597    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"]
Jul 30 23:04:42.769990 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.769950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-tmp-dir\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.769990 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.769991    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-model-cache\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.770240 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.770012    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8829b966-94c9-4841-935a-b1de9fef20e8-tls-certs\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.770240 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.770125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-home\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.770240 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.770173    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p24vr\" (UniqueName: \"kubernetes.io/projected/8829b966-94c9-4841-935a-b1de9fef20e8-kube-api-access-p24vr\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.770341 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.770235    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-dshm\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.871386 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.871347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p24vr\" (UniqueName: \"kubernetes.io/projected/8829b966-94c9-4841-935a-b1de9fef20e8-kube-api-access-p24vr\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.871386 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.871393    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-dshm\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.871648 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.871592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-tmp-dir\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.871648 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.871636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-model-cache\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.871749 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.871659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8829b966-94c9-4841-935a-b1de9fef20e8-tls-certs\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.871749 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.871715    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-home\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.871976 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.871949    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-tmp-dir\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.871976 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.871968    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-model-cache\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.872137 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.872015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-home\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.873977 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.873956    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-dshm\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.874258 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.874240    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8829b966-94c9-4841-935a-b1de9fef20e8-tls-certs\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:42.878866 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:42.878845    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p24vr\" (UniqueName: \"kubernetes.io/projected/8829b966-94c9-4841-935a-b1de9fef20e8-kube-api-access-p24vr\") pod \"e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") " pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:43.021214 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:43.021127    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:43.156883 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:43.156839    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"]
Jul 30 23:04:43.159409 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:04:43.159382    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8829b966_94c9_4841_935a_b1de9fef20e8.slice/crio-e69bd83af233016ca6256e66516ea8cdb99fba8531f7955392bf819dc8e64283 WatchSource:0}: Error finding container e69bd83af233016ca6256e66516ea8cdb99fba8531f7955392bf819dc8e64283: Status 404 returned error can't find the container with id e69bd83af233016ca6256e66516ea8cdb99fba8531f7955392bf819dc8e64283
Jul 30 23:04:43.601680 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:43.601638    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g" event={"ID":"8829b966-94c9-4841-935a-b1de9fef20e8","Type":"ContainerStarted","Data":"2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d"}
Jul 30 23:04:43.601680 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:43.601686    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g" event={"ID":"8829b966-94c9-4841-935a-b1de9fef20e8","Type":"ContainerStarted","Data":"e69bd83af233016ca6256e66516ea8cdb99fba8531f7955392bf819dc8e64283"}
Jul 30 23:04:43.617073 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:43.617005    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g" podStartSLOduration=1.6169865190000001 podStartE2EDuration="1.616986519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:04:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:04:43.616422681 +0000 UTC m=+1463.791386360" watchObservedRunningTime="2026-07-30 23:04:43.616986519 +0000 UTC m=+1463.791950256"
Jul 30 23:04:46.794092 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.794036    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"]
Jul 30 23:04:46.797890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.797868    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:46.800348 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.800327    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-59ba5ab4-kserve-self-signed-certs\""
Jul 30 23:04:46.806788 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.806765    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"]
Jul 30 23:04:46.908349 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.908314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-home\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:46.908529 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.908358    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-model-cache\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:46.908529 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.908376    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-dshm\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:46.908529 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.908415    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tmp-dir\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:46.908675 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.908528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tls-certs\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:46.908675 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:46.908589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twvcw\" (UniqueName: \"kubernetes.io/projected/4e506c0a-3ec8-483f-b001-a98e7b2735b7-kube-api-access-twvcw\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.009417 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.009383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twvcw\" (UniqueName: \"kubernetes.io/projected/4e506c0a-3ec8-483f-b001-a98e7b2735b7-kube-api-access-twvcw\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.009586 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.009432    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-home\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.009586 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.009473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-model-cache\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.009586 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.009490    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-dshm\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.009586 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.009505    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tmp-dir\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.009586 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.009562    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tls-certs\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.009952 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.009929    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-model-cache\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.009997 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.009974    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tmp-dir\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.010028 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.010004    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-home\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.012175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.012151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-dshm\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.012320 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.012302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tls-certs\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.018392 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.018367    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twvcw\" (UniqueName: \"kubernetes.io/projected/4e506c0a-3ec8-483f-b001-a98e7b2735b7-kube-api-access-twvcw\") pod \"e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") " pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.108333 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.108244    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:47.240548 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.240429    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"]
Jul 30 23:04:47.244097 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:04:47.243194    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e506c0a_3ec8_483f_b001_a98e7b2735b7.slice/crio-d5a73299594b353e1cba69472eec42a69963a66146d361a17c639e367ed86929 WatchSource:0}: Error finding container d5a73299594b353e1cba69472eec42a69963a66146d361a17c639e367ed86929: Status 404 returned error can't find the container with id d5a73299594b353e1cba69472eec42a69963a66146d361a17c639e367ed86929
Jul 30 23:04:47.621635 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.621594    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9" event={"ID":"4e506c0a-3ec8-483f-b001-a98e7b2735b7","Type":"ContainerStarted","Data":"91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38"}
Jul 30 23:04:47.621818 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.621646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9" event={"ID":"4e506c0a-3ec8-483f-b001-a98e7b2735b7","Type":"ContainerStarted","Data":"d5a73299594b353e1cba69472eec42a69963a66146d361a17c639e367ed86929"}
Jul 30 23:04:47.638108 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:47.638031    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9" podStartSLOduration=1.63801541 podStartE2EDuration="1.63801541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:04:47.637149646 +0000 UTC m=+1467.812113326" watchObservedRunningTime="2026-07-30 23:04:47.63801541 +0000 UTC m=+1467.812979088"
Jul 30 23:04:53.021725 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:53.021682    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:53.022145 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:53.021745    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:53.034182 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:53.034156    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:53.660217 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:53.660190    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:04:57.109074 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:57.108992    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:57.109074 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:57.109090    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:57.121599 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:57.121568    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:04:57.678388 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:04:57.678356    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:05:08.850662 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:08.850627    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"]
Jul 30 23:05:09.536205 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:09.536169    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"]
Jul 30 23:05:20.310198 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:20.310166    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:05:20.313688 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:20.313665    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:05:23.871985 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:23.871915    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g" podUID="8829b966-94c9-4841-935a-b1de9fef20e8" containerName="main" containerID="cri-o://2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d" gracePeriod=45
Jul 30 23:05:24.119323 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.119292    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:05:24.146016 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.145952    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-model-cache\") pod \"8829b966-94c9-4841-935a-b1de9fef20e8\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") "
Jul 30 23:05:24.146016 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.145987    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-home\") pod \"8829b966-94c9-4841-935a-b1de9fef20e8\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") "
Jul 30 23:05:24.146016 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.146012    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p24vr\" (UniqueName: \"kubernetes.io/projected/8829b966-94c9-4841-935a-b1de9fef20e8-kube-api-access-p24vr\") pod \"8829b966-94c9-4841-935a-b1de9fef20e8\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") "
Jul 30 23:05:24.146280 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.146044    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-dshm\") pod \"8829b966-94c9-4841-935a-b1de9fef20e8\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") "
Jul 30 23:05:24.146280 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.146109    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-tmp-dir\") pod \"8829b966-94c9-4841-935a-b1de9fef20e8\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") "
Jul 30 23:05:24.146280 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.146217    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8829b966-94c9-4841-935a-b1de9fef20e8-tls-certs\") pod \"8829b966-94c9-4841-935a-b1de9fef20e8\" (UID: \"8829b966-94c9-4841-935a-b1de9fef20e8\") "
Jul 30 23:05:24.146431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.146291    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "8829b966-94c9-4841-935a-b1de9fef20e8" (UID: "8829b966-94c9-4841-935a-b1de9fef20e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:05:24.146431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.146308    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-home" (OuterVolumeSpecName: "home") pod "8829b966-94c9-4841-935a-b1de9fef20e8" (UID: "8829b966-94c9-4841-935a-b1de9fef20e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:05:24.146545 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.146489    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8829b966-94c9-4841-935a-b1de9fef20e8" (UID: "8829b966-94c9-4841-935a-b1de9fef20e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:05:24.146545 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.146523    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.146545 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.146540    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.148408 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.148378    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8829b966-94c9-4841-935a-b1de9fef20e8-kube-api-access-p24vr" (OuterVolumeSpecName: "kube-api-access-p24vr") pod "8829b966-94c9-4841-935a-b1de9fef20e8" (UID: "8829b966-94c9-4841-935a-b1de9fef20e8"). InnerVolumeSpecName "kube-api-access-p24vr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:05:24.148634 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.148592    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-dshm" (OuterVolumeSpecName: "dshm") pod "8829b966-94c9-4841-935a-b1de9fef20e8" (UID: "8829b966-94c9-4841-935a-b1de9fef20e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:05:24.148709 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.148684    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8829b966-94c9-4841-935a-b1de9fef20e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8829b966-94c9-4841-935a-b1de9fef20e8" (UID: "8829b966-94c9-4841-935a-b1de9fef20e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:05:24.247631 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.247592    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8829b966-94c9-4841-935a-b1de9fef20e8-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.247631 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.247633    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p24vr\" (UniqueName: \"kubernetes.io/projected/8829b966-94c9-4841-935a-b1de9fef20e8-kube-api-access-p24vr\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.247823 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.247666    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.247823 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.247709    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8829b966-94c9-4841-935a-b1de9fef20e8-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.557299 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.557264    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9" podUID="4e506c0a-3ec8-483f-b001-a98e7b2735b7" containerName="main" containerID="cri-o://91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38" gracePeriod=45
Jul 30 23:05:24.793088 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.793025    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:05:24.803316 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.803285    2578 generic.go:356] "Generic (PLEG): container finished" podID="8829b966-94c9-4841-935a-b1de9fef20e8" containerID="2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d" exitCode=0
Jul 30 23:05:24.803470 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.803321    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"
Jul 30 23:05:24.803470 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.803377    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g" event={"ID":"8829b966-94c9-4841-935a-b1de9fef20e8","Type":"ContainerDied","Data":"2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d"}
Jul 30 23:05:24.803470 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.803418    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g" event={"ID":"8829b966-94c9-4841-935a-b1de9fef20e8","Type":"ContainerDied","Data":"e69bd83af233016ca6256e66516ea8cdb99fba8531f7955392bf819dc8e64283"}
Jul 30 23:05:24.803470 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.803437    2578 scope.go:117] "RemoveContainer" containerID="2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d"
Jul 30 23:05:24.804666 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.804644    2578 generic.go:356] "Generic (PLEG): container finished" podID="4e506c0a-3ec8-483f-b001-a98e7b2735b7" containerID="91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38" exitCode=0
Jul 30 23:05:24.804773 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.804691    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"
Jul 30 23:05:24.804773 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.804732    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9" event={"ID":"4e506c0a-3ec8-483f-b001-a98e7b2735b7","Type":"ContainerDied","Data":"91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38"}
Jul 30 23:05:24.804773 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.804767    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9" event={"ID":"4e506c0a-3ec8-483f-b001-a98e7b2735b7","Type":"ContainerDied","Data":"d5a73299594b353e1cba69472eec42a69963a66146d361a17c639e367ed86929"}
Jul 30 23:05:24.812183 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.812147    2578 scope.go:117] "RemoveContainer" containerID="2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d"
Jul 30 23:05:24.812874 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:05:24.812850    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d\": container with ID starting with 2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d not found: ID does not exist" containerID="2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d"
Jul 30 23:05:24.812974 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.812886    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d"} err="failed to get container status \"2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d\": rpc error: code = NotFound desc = could not find container \"2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d\": container with ID starting with 2387c493b6f3e69d402cf61188b87b9af977b00864d0e5e0b78199ff6b6d4a9d not found: ID does not exist"
Jul 30 23:05:24.812974 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.812910    2578 scope.go:117] "RemoveContainer" containerID="91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38"
Jul 30 23:05:24.822821 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.822799    2578 scope.go:117] "RemoveContainer" containerID="91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38"
Jul 30 23:05:24.823126 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:05:24.823104    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38\": container with ID starting with 91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38 not found: ID does not exist" containerID="91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38"
Jul 30 23:05:24.823208 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.823138    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38"} err="failed to get container status \"91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38\": rpc error: code = NotFound desc = could not find container \"91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38\": container with ID starting with 91e33cb7d0f77f4da33019f1d7c094748b1d058515dab2b749200ce1ce208e38 not found: ID does not exist"
Jul 30 23:05:24.823753 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.823731    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"]
Jul 30 23:05:24.826837 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.826818    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-59ba5ab4-kserve-6f4878cc9f-dwd7g"]
Jul 30 23:05:24.853525 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.853497    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-home\") pod \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") "
Jul 30 23:05:24.853672 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.853538    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tmp-dir\") pod \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") "
Jul 30 23:05:24.853672 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.853570    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tls-certs\") pod \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") "
Jul 30 23:05:24.853672 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.853602    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-model-cache\") pod \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") "
Jul 30 23:05:24.853672 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.853618    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twvcw\" (UniqueName: \"kubernetes.io/projected/4e506c0a-3ec8-483f-b001-a98e7b2735b7-kube-api-access-twvcw\") pod \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") "
Jul 30 23:05:24.853672 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.853659    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-dshm\") pod \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\" (UID: \"4e506c0a-3ec8-483f-b001-a98e7b2735b7\") "
Jul 30 23:05:24.853928 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.853807    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-home" (OuterVolumeSpecName: "home") pod "4e506c0a-3ec8-483f-b001-a98e7b2735b7" (UID: "4e506c0a-3ec8-483f-b001-a98e7b2735b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:05:24.853928 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.853890    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e506c0a-3ec8-483f-b001-a98e7b2735b7" (UID: "4e506c0a-3ec8-483f-b001-a98e7b2735b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:05:24.853928 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.853901    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e506c0a-3ec8-483f-b001-a98e7b2735b7" (UID: "4e506c0a-3ec8-483f-b001-a98e7b2735b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:05:24.855845 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.855817    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-dshm" (OuterVolumeSpecName: "dshm") pod "4e506c0a-3ec8-483f-b001-a98e7b2735b7" (UID: "4e506c0a-3ec8-483f-b001-a98e7b2735b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:05:24.855954 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.855857    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e506c0a-3ec8-483f-b001-a98e7b2735b7" (UID: "4e506c0a-3ec8-483f-b001-a98e7b2735b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:05:24.855954 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.855881    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e506c0a-3ec8-483f-b001-a98e7b2735b7-kube-api-access-twvcw" (OuterVolumeSpecName: "kube-api-access-twvcw") pod "4e506c0a-3ec8-483f-b001-a98e7b2735b7" (UID: "4e506c0a-3ec8-483f-b001-a98e7b2735b7"). InnerVolumeSpecName "kube-api-access-twvcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:05:24.954230 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.954195    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.954230 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.954223    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e506c0a-3ec8-483f-b001-a98e7b2735b7-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.954230 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.954232    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.954230 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.954241    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twvcw\" (UniqueName: \"kubernetes.io/projected/4e506c0a-3ec8-483f-b001-a98e7b2735b7-kube-api-access-twvcw\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.954685 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.954249    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:24.954685 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:24.954258    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e506c0a-3ec8-483f-b001-a98e7b2735b7-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:05:25.127287 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:25.127255    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"]
Jul 30 23:05:25.132866 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:25.132839    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-59ba5ab4-kserve-7cc8dff7ff-wgjk9"]
Jul 30 23:05:26.342011 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:26.341977    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e506c0a-3ec8-483f-b001-a98e7b2735b7" path="/var/lib/kubelet/pods/4e506c0a-3ec8-483f-b001-a98e7b2735b7/volumes"
Jul 30 23:05:26.342435 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:05:26.342420    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8829b966-94c9-4841-935a-b1de9fef20e8" path="/var/lib/kubelet/pods/8829b966-94c9-4841-935a-b1de9fef20e8/volumes"
Jul 30 23:09:15.695284 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.695242    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"]
Jul 30 23:09:15.695877 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.695748    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e506c0a-3ec8-483f-b001-a98e7b2735b7" containerName="main"
Jul 30 23:09:15.695877 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.695767    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e506c0a-3ec8-483f-b001-a98e7b2735b7" containerName="main"
Jul 30 23:09:15.695877 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.695783    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8829b966-94c9-4841-935a-b1de9fef20e8" containerName="main"
Jul 30 23:09:15.695877 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.695791    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8829b966-94c9-4841-935a-b1de9fef20e8" containerName="main"
Jul 30 23:09:15.695877 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.695873    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8829b966-94c9-4841-935a-b1de9fef20e8" containerName="main"
Jul 30 23:09:15.696169 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.695889    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e506c0a-3ec8-483f-b001-a98e7b2735b7" containerName="main"
Jul 30 23:09:15.698888 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.698867    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:15.701495 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.701472    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5066b352-dockercfg-2hn74\""
Jul 30 23:09:15.701616 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.701505    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5066b352\""
Jul 30 23:09:15.710386 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.710363    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"]
Jul 30 23:09:15.738750 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.738723    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"]
Jul 30 23:09:15.740422 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.740402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74851070-c662-4bf2-abfb-c3af795ae597-plugins-config-volume\") pod \"payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl\" (UID: \"74851070-c662-4bf2-abfb-c3af795ae597\") " pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:15.740496 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.740472    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqptq\" (UniqueName: \"kubernetes.io/projected/74851070-c662-4bf2-abfb-c3af795ae597-kube-api-access-sqptq\") pod \"payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl\" (UID: \"74851070-c662-4bf2-abfb-c3af795ae597\") " pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:15.741933 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.741914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:15.753208 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.753180    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"]
Jul 30 23:09:15.841572 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.841538    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqptq\" (UniqueName: \"kubernetes.io/projected/74851070-c662-4bf2-abfb-c3af795ae597-kube-api-access-sqptq\") pod \"payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl\" (UID: \"74851070-c662-4bf2-abfb-c3af795ae597\") " pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:15.841777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.841586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbqs2\" (UniqueName: \"kubernetes.io/projected/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-kube-api-access-vbqs2\") pod \"payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq\" (UID: \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:15.841777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.841646    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74851070-c662-4bf2-abfb-c3af795ae597-plugins-config-volume\") pod \"payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl\" (UID: \"74851070-c662-4bf2-abfb-c3af795ae597\") " pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:15.841777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.841681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq\" (UID: \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:15.842664 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.842639    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74851070-c662-4bf2-abfb-c3af795ae597-plugins-config-volume\") pod \"payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl\" (UID: \"74851070-c662-4bf2-abfb-c3af795ae597\") " pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:15.850402 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.850368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqptq\" (UniqueName: \"kubernetes.io/projected/74851070-c662-4bf2-abfb-c3af795ae597-kube-api-access-sqptq\") pod \"payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl\" (UID: \"74851070-c662-4bf2-abfb-c3af795ae597\") " pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:15.943254 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.943203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbqs2\" (UniqueName: \"kubernetes.io/projected/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-kube-api-access-vbqs2\") pod \"payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq\" (UID: \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:15.943443 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.943296    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq\" (UID: \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:15.944033 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.944008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq\" (UID: \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:15.951747 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:15.951684    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbqs2\" (UniqueName: \"kubernetes.io/projected/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-kube-api-access-vbqs2\") pod \"payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq\" (UID: \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:16.009488 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.009447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:16.052807 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.052759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:16.164757 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.164726    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"]
Jul 30 23:09:16.167113 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:09:16.167082    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74851070_c662_4bf2_abfb_c3af795ae597.slice/crio-7bfc6c9a44dc430f127368556f4417180a0578c6d1fcd12c6122c57c17f37eae WatchSource:0}: Error finding container 7bfc6c9a44dc430f127368556f4417180a0578c6d1fcd12c6122c57c17f37eae: Status 404 returned error can't find the container with id 7bfc6c9a44dc430f127368556f4417180a0578c6d1fcd12c6122c57c17f37eae
Jul 30 23:09:16.205231 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.205208    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"]
Jul 30 23:09:16.208485 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:09:16.208461    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8fe8ac_7401_4552_a10e_b3ba1ae7e3fe.slice/crio-e57c943ee434782bb7678b00f0437b0956cd3662a606b2ec4dd320850b99cf61 WatchSource:0}: Error finding container e57c943ee434782bb7678b00f0437b0956cd3662a606b2ec4dd320850b99cf61: Status 404 returned error can't find the container with id e57c943ee434782bb7678b00f0437b0956cd3662a606b2ec4dd320850b99cf61
Jul 30 23:09:16.854500 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.854466    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq" event={"ID":"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe","Type":"ContainerStarted","Data":"cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9"}
Jul 30 23:09:16.854919 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.854526    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq" event={"ID":"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe","Type":"ContainerStarted","Data":"e57c943ee434782bb7678b00f0437b0956cd3662a606b2ec4dd320850b99cf61"}
Jul 30 23:09:16.854919 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.854555    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:16.856177 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.856150    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl" event={"ID":"74851070-c662-4bf2-abfb-c3af795ae597","Type":"ContainerStarted","Data":"e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81"}
Jul 30 23:09:16.856298 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.856190    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl" event={"ID":"74851070-c662-4bf2-abfb-c3af795ae597","Type":"ContainerStarted","Data":"7bfc6c9a44dc430f127368556f4417180a0578c6d1fcd12c6122c57c17f37eae"}
Jul 30 23:09:16.856298 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.856221    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:16.871157 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.871105    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq" podStartSLOduration=1.87108655 podStartE2EDuration="1.87108655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:09:16.870196995 +0000 UTC m=+1737.045160677" watchObservedRunningTime="2026-07-30 23:09:16.87108655 +0000 UTC m=+1737.046050207"
Jul 30 23:09:16.887204 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:16.887156    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl" podStartSLOduration=1.887138279 podStartE2EDuration="1.887138279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:09:16.886683157 +0000 UTC m=+1737.061646840" watchObservedRunningTime="2026-07-30 23:09:16.887138279 +0000 UTC m=+1737.062101958"
Jul 30 23:09:27.862975 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:27.862943    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:27.863428 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:27.863025    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:45.628112 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.628069    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"]
Jul 30 23:09:45.628574 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.628483    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl" podUID="74851070-c662-4bf2-abfb-c3af795ae597" containerName="payload-processing" containerID="cri-o://e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81" gracePeriod=30
Jul 30 23:09:45.633934 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.633903    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"]
Jul 30 23:09:45.634430 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.634368    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq" podUID="9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe" containerName="payload-pre-processing" containerID="cri-o://cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9" gracePeriod=30
Jul 30 23:09:45.915835 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.915807    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:45.938020 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.937991    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:45.987596 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.987562    2578 generic.go:356] "Generic (PLEG): container finished" podID="9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe" containerID="cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9" exitCode=0
Jul 30 23:09:45.987753 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.987611    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"
Jul 30 23:09:45.987753 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.987651    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq" event={"ID":"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe","Type":"ContainerDied","Data":"cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9"}
Jul 30 23:09:45.987753 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.987724    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq" event={"ID":"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe","Type":"ContainerDied","Data":"e57c943ee434782bb7678b00f0437b0956cd3662a606b2ec4dd320850b99cf61"}
Jul 30 23:09:45.987875 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.987758    2578 scope.go:117] "RemoveContainer" containerID="cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9"
Jul 30 23:09:45.988938 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.988919    2578 generic.go:356] "Generic (PLEG): container finished" podID="74851070-c662-4bf2-abfb-c3af795ae597" containerID="e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81" exitCode=0
Jul 30 23:09:45.989042 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.988954    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl" event={"ID":"74851070-c662-4bf2-abfb-c3af795ae597","Type":"ContainerDied","Data":"e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81"}
Jul 30 23:09:45.989042 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.988966    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"
Jul 30 23:09:45.989042 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.988992    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl" event={"ID":"74851070-c662-4bf2-abfb-c3af795ae597","Type":"ContainerDied","Data":"7bfc6c9a44dc430f127368556f4417180a0578c6d1fcd12c6122c57c17f37eae"}
Jul 30 23:09:45.995924 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.995908    2578 scope.go:117] "RemoveContainer" containerID="cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9"
Jul 30 23:09:45.996262 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:09:45.996244    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9\": container with ID starting with cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9 not found: ID does not exist" containerID="cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9"
Jul 30 23:09:45.996313 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.996272    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9"} err="failed to get container status \"cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9\": rpc error: code = NotFound desc = could not find container \"cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9\": container with ID starting with cf36349dc46e6ec586bac19f965b80c2f38aeceb7eed2f19343777c72b0214a9 not found: ID does not exist"
Jul 30 23:09:45.996313 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:45.996291    2578 scope.go:117] "RemoveContainer" containerID="e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81"
Jul 30 23:09:46.003831 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.003808    2578 scope.go:117] "RemoveContainer" containerID="e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81"
Jul 30 23:09:46.004156 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:09:46.004137    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81\": container with ID starting with e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81 not found: ID does not exist" containerID="e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81"
Jul 30 23:09:46.004219 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.004163    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81"} err="failed to get container status \"e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81\": rpc error: code = NotFound desc = could not find container \"e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81\": container with ID starting with e12bff931d1118d15a48c9da2d8bac9a950f2f83b7952f80cf778ed3d9b40c81 not found: ID does not exist"
Jul 30 23:09:46.007477 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.007455    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74851070-c662-4bf2-abfb-c3af795ae597-plugins-config-volume\") pod \"74851070-c662-4bf2-abfb-c3af795ae597\" (UID: \"74851070-c662-4bf2-abfb-c3af795ae597\") "
Jul 30 23:09:46.007585 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.007514    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqptq\" (UniqueName: \"kubernetes.io/projected/74851070-c662-4bf2-abfb-c3af795ae597-kube-api-access-sqptq\") pod \"74851070-c662-4bf2-abfb-c3af795ae597\" (UID: \"74851070-c662-4bf2-abfb-c3af795ae597\") "
Jul 30 23:09:46.007808 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.007785    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74851070-c662-4bf2-abfb-c3af795ae597-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "74851070-c662-4bf2-abfb-c3af795ae597" (UID: "74851070-c662-4bf2-abfb-c3af795ae597"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:09:46.009559 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.009539    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74851070-c662-4bf2-abfb-c3af795ae597-kube-api-access-sqptq" (OuterVolumeSpecName: "kube-api-access-sqptq") pod "74851070-c662-4bf2-abfb-c3af795ae597" (UID: "74851070-c662-4bf2-abfb-c3af795ae597"). InnerVolumeSpecName "kube-api-access-sqptq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:09:46.108754 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.108715    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbqs2\" (UniqueName: \"kubernetes.io/projected/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-kube-api-access-vbqs2\") pod \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\" (UID: \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\") "
Jul 30 23:09:46.108960 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.108836    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-plugins-config-volume\") pod \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\" (UID: \"9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe\") "
Jul 30 23:09:46.108960 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.108951    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74851070-c662-4bf2-abfb-c3af795ae597-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:09:46.109116 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.108962    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqptq\" (UniqueName: \"kubernetes.io/projected/74851070-c662-4bf2-abfb-c3af795ae597-kube-api-access-sqptq\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:09:46.109243 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.109214    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe" (UID: "9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:09:46.110902 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.110880    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-kube-api-access-vbqs2" (OuterVolumeSpecName: "kube-api-access-vbqs2") pod "9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe" (UID: "9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe"). InnerVolumeSpecName "kube-api-access-vbqs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:09:46.209707 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.209661    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:09:46.209707 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.209705    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbqs2\" (UniqueName: \"kubernetes.io/projected/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe-kube-api-access-vbqs2\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:09:46.311333 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.311306    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"]
Jul 30 23:09:46.314902 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.314878    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5066b352-6d4667fdc4-mkqfl"]
Jul 30 23:09:46.324870 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.324847    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"]
Jul 30 23:09:46.326717 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.326697    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5066b352-65d977fdbd-bkfbq"]
Jul 30 23:09:46.342670 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.342640    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74851070-c662-4bf2-abfb-c3af795ae597" path="/var/lib/kubelet/pods/74851070-c662-4bf2-abfb-c3af795ae597/volumes"
Jul 30 23:09:46.343161 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:46.343142    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe" path="/var/lib/kubelet/pods/9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe/volumes"
Jul 30 23:09:50.042642 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.042612    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"]
Jul 30 23:09:50.043235 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.042956    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe" containerName="payload-pre-processing"
Jul 30 23:09:50.043235 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.042968    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe" containerName="payload-pre-processing"
Jul 30 23:09:50.043235 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.042980    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74851070-c662-4bf2-abfb-c3af795ae597" containerName="payload-processing"
Jul 30 23:09:50.043235 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.042986    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="74851070-c662-4bf2-abfb-c3af795ae597" containerName="payload-processing"
Jul 30 23:09:50.043235 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.043069    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="74851070-c662-4bf2-abfb-c3af795ae597" containerName="payload-processing"
Jul 30 23:09:50.043235 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.043081    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f8fe8ac-7401-4552-a10e-b3ba1ae7e3fe" containerName="payload-pre-processing"
Jul 30 23:09:50.047796 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.047774    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:09:50.050502 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.050472    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-0aa67640-dockercfg-n9xkn\""
Jul 30 23:09:50.050644 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.050612    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-0aa67640\""
Jul 30 23:09:50.056465 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.055836    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"]
Jul 30 23:09:50.063407 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.063383    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"]
Jul 30 23:09:50.066474 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.066458    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:09:50.075937 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.075910    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"]
Jul 30 23:09:50.142100 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.142028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc6809e5-05f5-47b6-b9c4-992f333c8387-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s\" (UID: \"fc6809e5-05f5-47b6-b9c4-992f333c8387\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:09:50.142279 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.142190    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79b8t\" (UniqueName: \"kubernetes.io/projected/fc6809e5-05f5-47b6-b9c4-992f333c8387-kube-api-access-79b8t\") pod \"payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s\" (UID: \"fc6809e5-05f5-47b6-b9c4-992f333c8387\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:09:50.243631 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.243592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc6809e5-05f5-47b6-b9c4-992f333c8387-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s\" (UID: \"fc6809e5-05f5-47b6-b9c4-992f333c8387\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:09:50.243857 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.243660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5f69\" (UniqueName: \"kubernetes.io/projected/dd9af361-c9f2-4011-9ea3-06b3b1809903-kube-api-access-k5f69\") pod \"payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62\" (UID: \"dd9af361-c9f2-4011-9ea3-06b3b1809903\") " pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:09:50.243857 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.243690    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79b8t\" (UniqueName: \"kubernetes.io/projected/fc6809e5-05f5-47b6-b9c4-992f333c8387-kube-api-access-79b8t\") pod \"payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s\" (UID: \"fc6809e5-05f5-47b6-b9c4-992f333c8387\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:09:50.243857 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.243742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd9af361-c9f2-4011-9ea3-06b3b1809903-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62\" (UID: \"dd9af361-c9f2-4011-9ea3-06b3b1809903\") " pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:09:50.244654 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.244622    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc6809e5-05f5-47b6-b9c4-992f333c8387-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s\" (UID: \"fc6809e5-05f5-47b6-b9c4-992f333c8387\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:09:50.252290 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.252262    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79b8t\" (UniqueName: \"kubernetes.io/projected/fc6809e5-05f5-47b6-b9c4-992f333c8387-kube-api-access-79b8t\") pod \"payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s\" (UID: \"fc6809e5-05f5-47b6-b9c4-992f333c8387\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:09:50.344847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.344762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5f69\" (UniqueName: \"kubernetes.io/projected/dd9af361-c9f2-4011-9ea3-06b3b1809903-kube-api-access-k5f69\") pod \"payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62\" (UID: \"dd9af361-c9f2-4011-9ea3-06b3b1809903\") " pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:09:50.344847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.344816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd9af361-c9f2-4011-9ea3-06b3b1809903-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62\" (UID: \"dd9af361-c9f2-4011-9ea3-06b3b1809903\") " pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:09:50.345582 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.345562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd9af361-c9f2-4011-9ea3-06b3b1809903-plugins-config-volume\") pod \"payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62\" (UID: \"dd9af361-c9f2-4011-9ea3-06b3b1809903\") " pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:09:50.353572 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.353545    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5f69\" (UniqueName: \"kubernetes.io/projected/dd9af361-c9f2-4011-9ea3-06b3b1809903-kube-api-access-k5f69\") pod \"payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62\" (UID: \"dd9af361-c9f2-4011-9ea3-06b3b1809903\") " pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:09:50.360345 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.360323    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:09:50.377823 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.377797    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:09:50.511770 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.511726    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"]
Jul 30 23:09:50.516844 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:09:50.516818    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc6809e5_05f5_47b6_b9c4_992f333c8387.slice/crio-cced1c30ff02a2e5274da9424899c67da50ea61bc5797b7ac809c402a0d4e108 WatchSource:0}: Error finding container cced1c30ff02a2e5274da9424899c67da50ea61bc5797b7ac809c402a0d4e108: Status 404 returned error can't find the container with id cced1c30ff02a2e5274da9424899c67da50ea61bc5797b7ac809c402a0d4e108
Jul 30 23:09:50.747456 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:50.747161    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"]
Jul 30 23:09:50.750341 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:09:50.750313    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9af361_c9f2_4011_9ea3_06b3b1809903.slice/crio-074dc2c6714423e5726df76d2853fae7611412d8a4be3168b2596ce5f398d62f WatchSource:0}: Error finding container 074dc2c6714423e5726df76d2853fae7611412d8a4be3168b2596ce5f398d62f: Status 404 returned error can't find the container with id 074dc2c6714423e5726df76d2853fae7611412d8a4be3168b2596ce5f398d62f
Jul 30 23:09:51.014713 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:51.014600    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s" event={"ID":"fc6809e5-05f5-47b6-b9c4-992f333c8387","Type":"ContainerStarted","Data":"adb6f563904d3206e4d1f10bed374af8bfdce0c34eb245aa88e5b917278ef091"}
Jul 30 23:09:51.014713 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:51.014671    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s" event={"ID":"fc6809e5-05f5-47b6-b9c4-992f333c8387","Type":"ContainerStarted","Data":"cced1c30ff02a2e5274da9424899c67da50ea61bc5797b7ac809c402a0d4e108"}
Jul 30 23:09:51.014949 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:51.014784    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:09:51.016158 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:51.016135    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62" event={"ID":"dd9af361-c9f2-4011-9ea3-06b3b1809903","Type":"ContainerStarted","Data":"6e077ab8077a33c7ba2bc44f0f923df3e79b06086aaa11acc7c96cb0ea17f722"}
Jul 30 23:09:51.016282 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:51.016172    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62" event={"ID":"dd9af361-c9f2-4011-9ea3-06b3b1809903","Type":"ContainerStarted","Data":"074dc2c6714423e5726df76d2853fae7611412d8a4be3168b2596ce5f398d62f"}
Jul 30 23:09:51.016324 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:51.016278    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:09:51.031368 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:51.031323    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s" podStartSLOduration=1.031305115 podStartE2EDuration="1.031305115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:09:51.0299371 +0000 UTC m=+1771.204900780" watchObservedRunningTime="2026-07-30 23:09:51.031305115 +0000 UTC m=+1771.206268806"
Jul 30 23:09:51.046348 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:09:51.046290    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62" podStartSLOduration=1.046274016 podStartE2EDuration="1.046274016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:09:51.045260427 +0000 UTC m=+1771.220224100" watchObservedRunningTime="2026-07-30 23:09:51.046274016 +0000 UTC m=+1771.221237698"
Jul 30 23:10:02.023487 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:02.023452    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:10:02.023899 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:02.023539    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:10:20.021686 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.021647    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"]
Jul 30 23:10:20.022183 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.021909    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62" podUID="dd9af361-c9f2-4011-9ea3-06b3b1809903" containerName="payload-processing" containerID="cri-o://6e077ab8077a33c7ba2bc44f0f923df3e79b06086aaa11acc7c96cb0ea17f722" gracePeriod=30
Jul 30 23:10:20.036333 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.036296    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"]
Jul 30 23:10:20.036636 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.036569    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s" podUID="fc6809e5-05f5-47b6-b9c4-992f333c8387" containerName="payload-pre-processing" containerID="cri-o://adb6f563904d3206e4d1f10bed374af8bfdce0c34eb245aa88e5b917278ef091" gracePeriod=30
Jul 30 23:10:20.160884 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.158394    2578 generic.go:356] "Generic (PLEG): container finished" podID="fc6809e5-05f5-47b6-b9c4-992f333c8387" containerID="adb6f563904d3206e4d1f10bed374af8bfdce0c34eb245aa88e5b917278ef091" exitCode=0
Jul 30 23:10:20.160884 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.158543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s" event={"ID":"fc6809e5-05f5-47b6-b9c4-992f333c8387","Type":"ContainerDied","Data":"adb6f563904d3206e4d1f10bed374af8bfdce0c34eb245aa88e5b917278ef091"}
Jul 30 23:10:20.162963 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.162930    2578 generic.go:356] "Generic (PLEG): container finished" podID="dd9af361-c9f2-4011-9ea3-06b3b1809903" containerID="6e077ab8077a33c7ba2bc44f0f923df3e79b06086aaa11acc7c96cb0ea17f722" exitCode=0
Jul 30 23:10:20.163102 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.163017    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62" event={"ID":"dd9af361-c9f2-4011-9ea3-06b3b1809903","Type":"ContainerDied","Data":"6e077ab8077a33c7ba2bc44f0f923df3e79b06086aaa11acc7c96cb0ea17f722"}
Jul 30 23:10:20.334302 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.334273    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:10:20.338343 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.338307    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:10:20.341082 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.341038    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:10:20.344603 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.344578    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:10:20.415612 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.415589    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd9af361-c9f2-4011-9ea3-06b3b1809903-plugins-config-volume\") pod \"dd9af361-c9f2-4011-9ea3-06b3b1809903\" (UID: \"dd9af361-c9f2-4011-9ea3-06b3b1809903\") "
Jul 30 23:10:20.415723 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.415647    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k5f69\" (UniqueName: \"kubernetes.io/projected/dd9af361-c9f2-4011-9ea3-06b3b1809903-kube-api-access-k5f69\") pod \"dd9af361-c9f2-4011-9ea3-06b3b1809903\" (UID: \"dd9af361-c9f2-4011-9ea3-06b3b1809903\") "
Jul 30 23:10:20.415723 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.415713    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc6809e5-05f5-47b6-b9c4-992f333c8387-plugins-config-volume\") pod \"fc6809e5-05f5-47b6-b9c4-992f333c8387\" (UID: \"fc6809e5-05f5-47b6-b9c4-992f333c8387\") "
Jul 30 23:10:20.415800 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.415749    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79b8t\" (UniqueName: \"kubernetes.io/projected/fc6809e5-05f5-47b6-b9c4-992f333c8387-kube-api-access-79b8t\") pod \"fc6809e5-05f5-47b6-b9c4-992f333c8387\" (UID: \"fc6809e5-05f5-47b6-b9c4-992f333c8387\") "
Jul 30 23:10:20.416066 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.416028    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd9af361-c9f2-4011-9ea3-06b3b1809903-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dd9af361-c9f2-4011-9ea3-06b3b1809903" (UID: "dd9af361-c9f2-4011-9ea3-06b3b1809903"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:10:20.416132 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.416109    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc6809e5-05f5-47b6-b9c4-992f333c8387-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fc6809e5-05f5-47b6-b9c4-992f333c8387" (UID: "fc6809e5-05f5-47b6-b9c4-992f333c8387"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:10:20.417984 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.417964    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd9af361-c9f2-4011-9ea3-06b3b1809903-kube-api-access-k5f69" (OuterVolumeSpecName: "kube-api-access-k5f69") pod "dd9af361-c9f2-4011-9ea3-06b3b1809903" (UID: "dd9af361-c9f2-4011-9ea3-06b3b1809903"). InnerVolumeSpecName "kube-api-access-k5f69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:10:20.418028 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.418001    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc6809e5-05f5-47b6-b9c4-992f333c8387-kube-api-access-79b8t" (OuterVolumeSpecName: "kube-api-access-79b8t") pod "fc6809e5-05f5-47b6-b9c4-992f333c8387" (UID: "fc6809e5-05f5-47b6-b9c4-992f333c8387"). InnerVolumeSpecName "kube-api-access-79b8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:10:20.517246 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.517203    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd9af361-c9f2-4011-9ea3-06b3b1809903-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:10:20.517246 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.517243    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5f69\" (UniqueName: \"kubernetes.io/projected/dd9af361-c9f2-4011-9ea3-06b3b1809903-kube-api-access-k5f69\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:10:20.517485 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.517257    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc6809e5-05f5-47b6-b9c4-992f333c8387-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:10:20.517485 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:20.517272    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79b8t\" (UniqueName: \"kubernetes.io/projected/fc6809e5-05f5-47b6-b9c4-992f333c8387-kube-api-access-79b8t\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:10:21.169962 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.169929    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"
Jul 30 23:10:21.169962 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.169952    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s" event={"ID":"fc6809e5-05f5-47b6-b9c4-992f333c8387","Type":"ContainerDied","Data":"cced1c30ff02a2e5274da9424899c67da50ea61bc5797b7ac809c402a0d4e108"}
Jul 30 23:10:21.170638 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.170031    2578 scope.go:117] "RemoveContainer" containerID="adb6f563904d3206e4d1f10bed374af8bfdce0c34eb245aa88e5b917278ef091"
Jul 30 23:10:21.171920 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.171894    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62" event={"ID":"dd9af361-c9f2-4011-9ea3-06b3b1809903","Type":"ContainerDied","Data":"074dc2c6714423e5726df76d2853fae7611412d8a4be3168b2596ce5f398d62f"}
Jul 30 23:10:21.172044 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.171972    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"
Jul 30 23:10:21.181854 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.181702    2578 scope.go:117] "RemoveContainer" containerID="6e077ab8077a33c7ba2bc44f0f923df3e79b06086aaa11acc7c96cb0ea17f722"
Jul 30 23:10:21.192221 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.192194    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"]
Jul 30 23:10:21.196114 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.196092    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-0aa67640-7b488776-9z45s"]
Jul 30 23:10:21.206159 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.206127    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"]
Jul 30 23:10:21.210434 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:21.210409    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-0aa67640-6d755bc6df-mqw62"]
Jul 30 23:10:22.342120 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:22.342090    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd9af361-c9f2-4011-9ea3-06b3b1809903" path="/var/lib/kubelet/pods/dd9af361-c9f2-4011-9ea3-06b3b1809903/volumes"
Jul 30 23:10:22.342514 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:22.342431    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc6809e5-05f5-47b6-b9c4-992f333c8387" path="/var/lib/kubelet/pods/fc6809e5-05f5-47b6-b9c4-992f333c8387/volumes"
Jul 30 23:10:25.430718 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.430680    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"]
Jul 30 23:10:25.431109 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.430997    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd9af361-c9f2-4011-9ea3-06b3b1809903" containerName="payload-processing"
Jul 30 23:10:25.431109 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.431008    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd9af361-c9f2-4011-9ea3-06b3b1809903" containerName="payload-processing"
Jul 30 23:10:25.431109 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.431020    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc6809e5-05f5-47b6-b9c4-992f333c8387" containerName="payload-pre-processing"
Jul 30 23:10:25.431109 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.431026    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc6809e5-05f5-47b6-b9c4-992f333c8387" containerName="payload-pre-processing"
Jul 30 23:10:25.431109 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.431100    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc6809e5-05f5-47b6-b9c4-992f333c8387" containerName="payload-pre-processing"
Jul 30 23:10:25.431109 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.431108    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd9af361-c9f2-4011-9ea3-06b3b1809903" containerName="payload-processing"
Jul 30 23:10:25.435322 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.435305    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:25.442105 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.442085    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-b7dedfa4\""
Jul 30 23:10:25.451563 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.451540    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"]
Jul 30 23:10:25.507595 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.507561    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"]
Jul 30 23:10:25.510881 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.510862    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:25.513627 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.513606    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-b7dedfa4-dockercfg-5bsn2\""
Jul 30 23:10:25.522219 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.522199    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"]
Jul 30 23:10:25.562002 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.561969    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4\" (UID: \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:25.562196 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.562082    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cr64\" (UniqueName: \"kubernetes.io/projected/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-kube-api-access-7cr64\") pod \"payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4\" (UID: \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:25.663190 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.663155    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cr64\" (UniqueName: \"kubernetes.io/projected/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-kube-api-access-7cr64\") pod \"payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4\" (UID: \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:25.663366 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.663206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4nlh\" (UniqueName: \"kubernetes.io/projected/d3262fdc-8e78-43be-b74f-ebf82157d68b-kube-api-access-m4nlh\") pod \"payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5\" (UID: \"d3262fdc-8e78-43be-b74f-ebf82157d68b\") " pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:25.663366 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.663235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4\" (UID: \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:25.663366 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.663281    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3262fdc-8e78-43be-b74f-ebf82157d68b-plugins-config-volume\") pod \"payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5\" (UID: \"d3262fdc-8e78-43be-b74f-ebf82157d68b\") " pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:25.663940 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.663922    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4\" (UID: \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:25.671936 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.671906    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cr64\" (UniqueName: \"kubernetes.io/projected/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-kube-api-access-7cr64\") pod \"payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4\" (UID: \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:25.746299 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.746264    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:25.764594 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.764556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3262fdc-8e78-43be-b74f-ebf82157d68b-plugins-config-volume\") pod \"payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5\" (UID: \"d3262fdc-8e78-43be-b74f-ebf82157d68b\") " pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:25.764734 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.764657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4nlh\" (UniqueName: \"kubernetes.io/projected/d3262fdc-8e78-43be-b74f-ebf82157d68b-kube-api-access-m4nlh\") pod \"payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5\" (UID: \"d3262fdc-8e78-43be-b74f-ebf82157d68b\") " pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:25.765235 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.765214    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3262fdc-8e78-43be-b74f-ebf82157d68b-plugins-config-volume\") pod \"payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5\" (UID: \"d3262fdc-8e78-43be-b74f-ebf82157d68b\") " pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:25.772774 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.772752    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4nlh\" (UniqueName: \"kubernetes.io/projected/d3262fdc-8e78-43be-b74f-ebf82157d68b-kube-api-access-m4nlh\") pod \"payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5\" (UID: \"d3262fdc-8e78-43be-b74f-ebf82157d68b\") " pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:25.823010 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.822679    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:25.921212 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.921082    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"]
Jul 30 23:10:25.923918 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:10:25.923884    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbdb9a98_d0d7_454b_aef1_69a50e9b5530.slice/crio-31dd5e12d25de54bfa2db393b41f69bf8980f93be93a1770552d47e611e74f2b WatchSource:0}: Error finding container 31dd5e12d25de54bfa2db393b41f69bf8980f93be93a1770552d47e611e74f2b: Status 404 returned error can't find the container with id 31dd5e12d25de54bfa2db393b41f69bf8980f93be93a1770552d47e611e74f2b
Jul 30 23:10:25.975516 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:25.975489    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"]
Jul 30 23:10:25.984118 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:10:25.984091    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3262fdc_8e78_43be_b74f_ebf82157d68b.slice/crio-86d50633c415898b242f8c36f5ec82fab0044fba35d2a4c01f04e5ebf66281c3 WatchSource:0}: Error finding container 86d50633c415898b242f8c36f5ec82fab0044fba35d2a4c01f04e5ebf66281c3: Status 404 returned error can't find the container with id 86d50633c415898b242f8c36f5ec82fab0044fba35d2a4c01f04e5ebf66281c3
Jul 30 23:10:26.197415 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:26.197375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5" event={"ID":"d3262fdc-8e78-43be-b74f-ebf82157d68b","Type":"ContainerStarted","Data":"8aa0546b67331d2a6dcd0e8da23176ad9d5adab5e52fd85ca3a61f42d91f8552"}
Jul 30 23:10:26.197608 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:26.197439    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5" event={"ID":"d3262fdc-8e78-43be-b74f-ebf82157d68b","Type":"ContainerStarted","Data":"86d50633c415898b242f8c36f5ec82fab0044fba35d2a4c01f04e5ebf66281c3"}
Jul 30 23:10:26.197608 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:26.197463    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:26.198929 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:26.198908    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4" event={"ID":"dbdb9a98-d0d7-454b-aef1-69a50e9b5530","Type":"ContainerStarted","Data":"71eec36687eea04dac016cd2ee982ab131e3cf5e1dd6de4df78adb3f2cec2464"}
Jul 30 23:10:26.199085 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:26.198944    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4" event={"ID":"dbdb9a98-d0d7-454b-aef1-69a50e9b5530","Type":"ContainerStarted","Data":"31dd5e12d25de54bfa2db393b41f69bf8980f93be93a1770552d47e611e74f2b"}
Jul 30 23:10:26.199085 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:26.198978    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:26.211831 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:26.211761    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5" podStartSLOduration=1.211744519 podStartE2EDuration="1.211744519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:10:26.211315411 +0000 UTC m=+1806.386279090" watchObservedRunningTime="2026-07-30 23:10:26.211744519 +0000 UTC m=+1806.386708198"
Jul 30 23:10:26.226985 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:26.226940    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4" podStartSLOduration=1.226925185 podStartE2EDuration="1.226925185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:10:26.225428966 +0000 UTC m=+1806.400392645" watchObservedRunningTime="2026-07-30 23:10:26.226925185 +0000 UTC m=+1806.401888863"
Jul 30 23:10:37.206774 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:37.206741    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:37.207182 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:37.206814    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:55.185994 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.185961    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"]
Jul 30 23:10:55.186547 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.186191    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5" podUID="d3262fdc-8e78-43be-b74f-ebf82157d68b" containerName="payload-processing" containerID="cri-o://8aa0546b67331d2a6dcd0e8da23176ad9d5adab5e52fd85ca3a61f42d91f8552" gracePeriod=30
Jul 30 23:10:55.191653 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.191548    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"]
Jul 30 23:10:55.192079 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.191788    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4" podUID="dbdb9a98-d0d7-454b-aef1-69a50e9b5530" containerName="payload-pre-processing" containerID="cri-o://71eec36687eea04dac016cd2ee982ab131e3cf5e1dd6de4df78adb3f2cec2464" gracePeriod=30
Jul 30 23:10:55.343561 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.343519    2578 generic.go:356] "Generic (PLEG): container finished" podID="dbdb9a98-d0d7-454b-aef1-69a50e9b5530" containerID="71eec36687eea04dac016cd2ee982ab131e3cf5e1dd6de4df78adb3f2cec2464" exitCode=0
Jul 30 23:10:55.343738 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.343588    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4" event={"ID":"dbdb9a98-d0d7-454b-aef1-69a50e9b5530","Type":"ContainerDied","Data":"71eec36687eea04dac016cd2ee982ab131e3cf5e1dd6de4df78adb3f2cec2464"}
Jul 30 23:10:55.345693 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.345665    2578 generic.go:356] "Generic (PLEG): container finished" podID="d3262fdc-8e78-43be-b74f-ebf82157d68b" containerID="8aa0546b67331d2a6dcd0e8da23176ad9d5adab5e52fd85ca3a61f42d91f8552" exitCode=0
Jul 30 23:10:55.345835 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.345713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5" event={"ID":"d3262fdc-8e78-43be-b74f-ebf82157d68b","Type":"ContainerDied","Data":"8aa0546b67331d2a6dcd0e8da23176ad9d5adab5e52fd85ca3a61f42d91f8552"}
Jul 30 23:10:55.447025 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.447001    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:55.484008 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.483978    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:55.527870 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.527839    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m4nlh\" (UniqueName: \"kubernetes.io/projected/d3262fdc-8e78-43be-b74f-ebf82157d68b-kube-api-access-m4nlh\") pod \"d3262fdc-8e78-43be-b74f-ebf82157d68b\" (UID: \"d3262fdc-8e78-43be-b74f-ebf82157d68b\") "
Jul 30 23:10:55.528030 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.527893    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-plugins-config-volume\") pod \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\" (UID: \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\") "
Jul 30 23:10:55.528030 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.527929    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3262fdc-8e78-43be-b74f-ebf82157d68b-plugins-config-volume\") pod \"d3262fdc-8e78-43be-b74f-ebf82157d68b\" (UID: \"d3262fdc-8e78-43be-b74f-ebf82157d68b\") "
Jul 30 23:10:55.528030 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.527967    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cr64\" (UniqueName: \"kubernetes.io/projected/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-kube-api-access-7cr64\") pod \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\" (UID: \"dbdb9a98-d0d7-454b-aef1-69a50e9b5530\") "
Jul 30 23:10:55.528392 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.528367    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3262fdc-8e78-43be-b74f-ebf82157d68b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d3262fdc-8e78-43be-b74f-ebf82157d68b" (UID: "d3262fdc-8e78-43be-b74f-ebf82157d68b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:10:55.528392 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.528372    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dbdb9a98-d0d7-454b-aef1-69a50e9b5530" (UID: "dbdb9a98-d0d7-454b-aef1-69a50e9b5530"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:10:55.530145 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.530123    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3262fdc-8e78-43be-b74f-ebf82157d68b-kube-api-access-m4nlh" (OuterVolumeSpecName: "kube-api-access-m4nlh") pod "d3262fdc-8e78-43be-b74f-ebf82157d68b" (UID: "d3262fdc-8e78-43be-b74f-ebf82157d68b"). InnerVolumeSpecName "kube-api-access-m4nlh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:10:55.530269 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.530250    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-kube-api-access-7cr64" (OuterVolumeSpecName: "kube-api-access-7cr64") pod "dbdb9a98-d0d7-454b-aef1-69a50e9b5530" (UID: "dbdb9a98-d0d7-454b-aef1-69a50e9b5530"). InnerVolumeSpecName "kube-api-access-7cr64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:10:55.629594 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.629552    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d3262fdc-8e78-43be-b74f-ebf82157d68b-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:10:55.629594 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.629586    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cr64\" (UniqueName: \"kubernetes.io/projected/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-kube-api-access-7cr64\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:10:55.629594 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.629599    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4nlh\" (UniqueName: \"kubernetes.io/projected/d3262fdc-8e78-43be-b74f-ebf82157d68b-kube-api-access-m4nlh\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:10:55.629823 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:55.629611    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dbdb9a98-d0d7-454b-aef1-69a50e9b5530-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:10:56.353181 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.353154    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"
Jul 30 23:10:56.353581 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.353181    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4" event={"ID":"dbdb9a98-d0d7-454b-aef1-69a50e9b5530","Type":"ContainerDied","Data":"31dd5e12d25de54bfa2db393b41f69bf8980f93be93a1770552d47e611e74f2b"}
Jul 30 23:10:56.353581 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.353249    2578 scope.go:117] "RemoveContainer" containerID="71eec36687eea04dac016cd2ee982ab131e3cf5e1dd6de4df78adb3f2cec2464"
Jul 30 23:10:56.355014 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.354992    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5" event={"ID":"d3262fdc-8e78-43be-b74f-ebf82157d68b","Type":"ContainerDied","Data":"86d50633c415898b242f8c36f5ec82fab0044fba35d2a4c01f04e5ebf66281c3"}
Jul 30 23:10:56.355159 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.355141    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"
Jul 30 23:10:56.362248 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.362230    2578 scope.go:117] "RemoveContainer" containerID="8aa0546b67331d2a6dcd0e8da23176ad9d5adab5e52fd85ca3a61f42d91f8552"
Jul 30 23:10:56.372121 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.372088    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"]
Jul 30 23:10:56.375906 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.375883    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-b7dedfa4-58d88ffbfb-6dln4"]
Jul 30 23:10:56.386096 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.386073    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"]
Jul 30 23:10:56.387634 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:56.387615    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-b7dedfa4-596cb848b-46jg5"]
Jul 30 23:10:58.341984 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:58.341943    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3262fdc-8e78-43be-b74f-ebf82157d68b" path="/var/lib/kubelet/pods/d3262fdc-8e78-43be-b74f-ebf82157d68b/volumes"
Jul 30 23:10:58.342527 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:10:58.342466    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbdb9a98-d0d7-454b-aef1-69a50e9b5530" path="/var/lib/kubelet/pods/dbdb9a98-d0d7-454b-aef1-69a50e9b5530/volumes"
Jul 30 23:11:01.338351 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.338316    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"]
Jul 30 23:11:01.338731 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.338703    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbdb9a98-d0d7-454b-aef1-69a50e9b5530" containerName="payload-pre-processing"
Jul 30 23:11:01.338731 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.338717    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbdb9a98-d0d7-454b-aef1-69a50e9b5530" containerName="payload-pre-processing"
Jul 30 23:11:01.338731 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.338729    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3262fdc-8e78-43be-b74f-ebf82157d68b" containerName="payload-processing"
Jul 30 23:11:01.338824 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.338734    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3262fdc-8e78-43be-b74f-ebf82157d68b" containerName="payload-processing"
Jul 30 23:11:01.338824 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.338785    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3262fdc-8e78-43be-b74f-ebf82157d68b" containerName="payload-processing"
Jul 30 23:11:01.338824 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.338796    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbdb9a98-d0d7-454b-aef1-69a50e9b5530" containerName="payload-pre-processing"
Jul 30 23:11:01.343473 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.343446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:01.349420 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.349395    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-94a93f51\""
Jul 30 23:11:01.350808 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.350782    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-94a93f51-dockercfg-d4znb\""
Jul 30 23:11:01.353259 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.353239    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"]
Jul 30 23:11:01.373822 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.373786    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"]
Jul 30 23:11:01.378568 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.378539    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:01.386878 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.386852    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"]
Jul 30 23:11:01.483130 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.483091    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d\" (UID: \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:01.483130 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.483133    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-plugins-config-volume\") pod \"payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct\" (UID: \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\") " pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:01.483350 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.483236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75tjt\" (UniqueName: \"kubernetes.io/projected/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-kube-api-access-75tjt\") pod \"payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d\" (UID: \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:01.483350 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.483310    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4mhd\" (UniqueName: \"kubernetes.io/projected/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-kube-api-access-r4mhd\") pod \"payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct\" (UID: \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\") " pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:01.584576 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.584536    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d\" (UID: \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:01.584576 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.584581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-plugins-config-volume\") pod \"payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct\" (UID: \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\") " pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:01.584897 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.584625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75tjt\" (UniqueName: \"kubernetes.io/projected/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-kube-api-access-75tjt\") pod \"payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d\" (UID: \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:01.584897 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.584653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4mhd\" (UniqueName: \"kubernetes.io/projected/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-kube-api-access-r4mhd\") pod \"payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct\" (UID: \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\") " pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:01.585366 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.585344    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-plugins-config-volume\") pod \"payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct\" (UID: \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\") " pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:01.585416 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.585376    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d\" (UID: \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:01.593457 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.593397    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75tjt\" (UniqueName: \"kubernetes.io/projected/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-kube-api-access-75tjt\") pod \"payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d\" (UID: \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:01.593457 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.593422    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4mhd\" (UniqueName: \"kubernetes.io/projected/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-kube-api-access-r4mhd\") pod \"payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct\" (UID: \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\") " pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:01.655962 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.655921    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:01.689444 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.689392    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:01.814172 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.814137    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"]
Jul 30 23:11:01.817026 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:11:01.816988    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d512343_ce8f_49d9_bd5c_eb3a576e8ad5.slice/crio-a0a3b872ff125a9762de5f579baf60086796eb20360e5ee914a39264c9236a49 WatchSource:0}: Error finding container a0a3b872ff125a9762de5f579baf60086796eb20360e5ee914a39264c9236a49: Status 404 returned error can't find the container with id a0a3b872ff125a9762de5f579baf60086796eb20360e5ee914a39264c9236a49
Jul 30 23:11:01.846467 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:01.846436    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"]
Jul 30 23:11:01.856297 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:11:01.856264    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261ea6b2_86f9_4928_bc39_7a32dab4fd9a.slice/crio-b6831d6974fb087ca078e1a00978f7d3e365d83c68bd8e17c8129bc79806496f WatchSource:0}: Error finding container b6831d6974fb087ca078e1a00978f7d3e365d83c68bd8e17c8129bc79806496f: Status 404 returned error can't find the container with id b6831d6974fb087ca078e1a00978f7d3e365d83c68bd8e17c8129bc79806496f
Jul 30 23:11:02.386577 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:02.386531    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d" event={"ID":"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5","Type":"ContainerStarted","Data":"77f9948aad1607ad1c671996432880c2a95711bdd4cfb5dd5d0f0898ed5c8f4a"}
Jul 30 23:11:02.387087 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:02.386705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:02.387087 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:02.386731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d" event={"ID":"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5","Type":"ContainerStarted","Data":"a0a3b872ff125a9762de5f579baf60086796eb20360e5ee914a39264c9236a49"}
Jul 30 23:11:02.388142 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:02.388116    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct" event={"ID":"261ea6b2-86f9-4928-bc39-7a32dab4fd9a","Type":"ContainerStarted","Data":"ee045fdff58c1f3bb8d6b0b87b024f10187c2a50099afc6a81ba61fcbc7d8eea"}
Jul 30 23:11:02.388261 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:02.388159    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct" event={"ID":"261ea6b2-86f9-4928-bc39-7a32dab4fd9a","Type":"ContainerStarted","Data":"b6831d6974fb087ca078e1a00978f7d3e365d83c68bd8e17c8129bc79806496f"}
Jul 30 23:11:02.388261 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:02.388189    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:02.401340 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:02.401289    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d" podStartSLOduration=1.401271906 podStartE2EDuration="1.401271906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:11:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:11:02.400083832 +0000 UTC m=+1842.575047507" watchObservedRunningTime="2026-07-30 23:11:02.401271906 +0000 UTC m=+1842.576235649"
Jul 30 23:11:02.416210 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:02.416146    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct" podStartSLOduration=1.416125061 podStartE2EDuration="1.416125061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:11:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:11:02.414384042 +0000 UTC m=+1842.589347723" watchObservedRunningTime="2026-07-30 23:11:02.416125061 +0000 UTC m=+1842.591088741"
Jul 30 23:11:13.395429 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:13.395384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:13.395896 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:13.395625    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:31.294143 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.294108    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"]
Jul 30 23:11:31.294640 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.294361    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct" podUID="261ea6b2-86f9-4928-bc39-7a32dab4fd9a" containerName="payload-processing" containerID="cri-o://ee045fdff58c1f3bb8d6b0b87b024f10187c2a50099afc6a81ba61fcbc7d8eea" gracePeriod=30
Jul 30 23:11:31.301701 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.301658    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"]
Jul 30 23:11:31.301941 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.301891    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d" podUID="5d512343-ce8f-49d9-bd5c-eb3a576e8ad5" containerName="payload-pre-processing" containerID="cri-o://77f9948aad1607ad1c671996432880c2a95711bdd4cfb5dd5d0f0898ed5c8f4a" gracePeriod=30
Jul 30 23:11:31.528326 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.528287    2578 generic.go:356] "Generic (PLEG): container finished" podID="261ea6b2-86f9-4928-bc39-7a32dab4fd9a" containerID="ee045fdff58c1f3bb8d6b0b87b024f10187c2a50099afc6a81ba61fcbc7d8eea" exitCode=0
Jul 30 23:11:31.528503 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.528355    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct" event={"ID":"261ea6b2-86f9-4928-bc39-7a32dab4fd9a","Type":"ContainerDied","Data":"ee045fdff58c1f3bb8d6b0b87b024f10187c2a50099afc6a81ba61fcbc7d8eea"}
Jul 30 23:11:31.531062 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.531015    2578 generic.go:356] "Generic (PLEG): container finished" podID="5d512343-ce8f-49d9-bd5c-eb3a576e8ad5" containerID="77f9948aad1607ad1c671996432880c2a95711bdd4cfb5dd5d0f0898ed5c8f4a" exitCode=0
Jul 30 23:11:31.531208 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.531125    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d" event={"ID":"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5","Type":"ContainerDied","Data":"77f9948aad1607ad1c671996432880c2a95711bdd4cfb5dd5d0f0898ed5c8f4a"}
Jul 30 23:11:31.590774 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.590745    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:31.593718 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.593693    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:31.636758 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.636725    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-plugins-config-volume\") pod \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\" (UID: \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\") "
Jul 30 23:11:31.636954 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.636799    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75tjt\" (UniqueName: \"kubernetes.io/projected/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-kube-api-access-75tjt\") pod \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\" (UID: \"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5\") "
Jul 30 23:11:31.636954 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.636831    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-plugins-config-volume\") pod \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\" (UID: \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\") "
Jul 30 23:11:31.636954 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.636889    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4mhd\" (UniqueName: \"kubernetes.io/projected/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-kube-api-access-r4mhd\") pod \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\" (UID: \"261ea6b2-86f9-4928-bc39-7a32dab4fd9a\") "
Jul 30 23:11:31.637253 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.637225    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5d512343-ce8f-49d9-bd5c-eb3a576e8ad5" (UID: "5d512343-ce8f-49d9-bd5c-eb3a576e8ad5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:11:31.637321 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.637244    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "261ea6b2-86f9-4928-bc39-7a32dab4fd9a" (UID: "261ea6b2-86f9-4928-bc39-7a32dab4fd9a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:11:31.639179 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.639147    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-kube-api-access-r4mhd" (OuterVolumeSpecName: "kube-api-access-r4mhd") pod "261ea6b2-86f9-4928-bc39-7a32dab4fd9a" (UID: "261ea6b2-86f9-4928-bc39-7a32dab4fd9a"). InnerVolumeSpecName "kube-api-access-r4mhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:11:31.639268 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.639181    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-kube-api-access-75tjt" (OuterVolumeSpecName: "kube-api-access-75tjt") pod "5d512343-ce8f-49d9-bd5c-eb3a576e8ad5" (UID: "5d512343-ce8f-49d9-bd5c-eb3a576e8ad5"). InnerVolumeSpecName "kube-api-access-75tjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:11:31.737621 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.737574    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:11:31.737621 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.737612    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75tjt\" (UniqueName: \"kubernetes.io/projected/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5-kube-api-access-75tjt\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:11:31.737621 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.737622    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:11:31.737621 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:31.737630    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4mhd\" (UniqueName: \"kubernetes.io/projected/261ea6b2-86f9-4928-bc39-7a32dab4fd9a-kube-api-access-r4mhd\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:11:32.536552 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.536518    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"
Jul 30 23:11:32.537030 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.536552    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d" event={"ID":"5d512343-ce8f-49d9-bd5c-eb3a576e8ad5","Type":"ContainerDied","Data":"a0a3b872ff125a9762de5f579baf60086796eb20360e5ee914a39264c9236a49"}
Jul 30 23:11:32.537030 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.536615    2578 scope.go:117] "RemoveContainer" containerID="77f9948aad1607ad1c671996432880c2a95711bdd4cfb5dd5d0f0898ed5c8f4a"
Jul 30 23:11:32.538022 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.538004    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"
Jul 30 23:11:32.538305 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.538007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct" event={"ID":"261ea6b2-86f9-4928-bc39-7a32dab4fd9a","Type":"ContainerDied","Data":"b6831d6974fb087ca078e1a00978f7d3e365d83c68bd8e17c8129bc79806496f"}
Jul 30 23:11:32.547900 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.547879    2578 scope.go:117] "RemoveContainer" containerID="ee045fdff58c1f3bb8d6b0b87b024f10187c2a50099afc6a81ba61fcbc7d8eea"
Jul 30 23:11:32.558145 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.558105    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"]
Jul 30 23:11:32.563296 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.563273    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-94a93f51-dd68876d-grx7d"]
Jul 30 23:11:32.573497 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.573473    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"]
Jul 30 23:11:32.576649 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:32.576626    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-94a93f51-5b588f8f5d-xdvct"]
Jul 30 23:11:34.342698 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:34.342654    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="261ea6b2-86f9-4928-bc39-7a32dab4fd9a" path="/var/lib/kubelet/pods/261ea6b2-86f9-4928-bc39-7a32dab4fd9a/volumes"
Jul 30 23:11:34.343171 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:11:34.343153    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d512343-ce8f-49d9-bd5c-eb3a576e8ad5" path="/var/lib/kubelet/pods/5d512343-ce8f-49d9-bd5c-eb3a576e8ad5/volumes"
Jul 30 23:13:35.048598 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.048559    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"]
Jul 30 23:13:35.050890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.048905    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261ea6b2-86f9-4928-bc39-7a32dab4fd9a" containerName="payload-processing"
Jul 30 23:13:35.050890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.048918    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="261ea6b2-86f9-4928-bc39-7a32dab4fd9a" containerName="payload-processing"
Jul 30 23:13:35.050890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.048931    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d512343-ce8f-49d9-bd5c-eb3a576e8ad5" containerName="payload-pre-processing"
Jul 30 23:13:35.050890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.048940    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d512343-ce8f-49d9-bd5c-eb3a576e8ad5" containerName="payload-pre-processing"
Jul 30 23:13:35.050890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.048998    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="261ea6b2-86f9-4928-bc39-7a32dab4fd9a" containerName="payload-processing"
Jul 30 23:13:35.050890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.049023    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d512343-ce8f-49d9-bd5c-eb3a576e8ad5" containerName="payload-pre-processing"
Jul 30 23:13:35.051805 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.051790    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:35.054592 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.054573    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-85a1df81-dockercfg-8c9f8\""
Jul 30 23:13:35.055070 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.055043    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-85a1df81\""
Jul 30 23:13:35.063111 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.063085    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"]
Jul 30 23:13:35.083306 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.083272    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rffzj\" (UniqueName: \"kubernetes.io/projected/6822df7f-14cc-4b2c-a375-5946553b83b7-kube-api-access-rffzj\") pod \"payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv\" (UID: \"6822df7f-14cc-4b2c-a375-5946553b83b7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:35.083466 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.083329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6822df7f-14cc-4b2c-a375-5946553b83b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv\" (UID: \"6822df7f-14cc-4b2c-a375-5946553b83b7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:35.087513 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.087488    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"]
Jul 30 23:13:35.090795 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.090778    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:13:35.105426 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.105393    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"]
Jul 30 23:13:35.184670 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.184633    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6822df7f-14cc-4b2c-a375-5946553b83b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv\" (UID: \"6822df7f-14cc-4b2c-a375-5946553b83b7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:35.184848 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.184694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4975719-cc61-4442-87fe-f5e8cb580ac0-plugins-config-volume\") pod \"payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt\" (UID: \"c4975719-cc61-4442-87fe-f5e8cb580ac0\") " pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:13:35.184848 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.184762    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rffzj\" (UniqueName: \"kubernetes.io/projected/6822df7f-14cc-4b2c-a375-5946553b83b7-kube-api-access-rffzj\") pod \"payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv\" (UID: \"6822df7f-14cc-4b2c-a375-5946553b83b7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:35.184848 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.184791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbppv\" (UniqueName: \"kubernetes.io/projected/c4975719-cc61-4442-87fe-f5e8cb580ac0-kube-api-access-jbppv\") pod \"payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt\" (UID: \"c4975719-cc61-4442-87fe-f5e8cb580ac0\") " pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:13:35.185369 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.185351    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6822df7f-14cc-4b2c-a375-5946553b83b7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv\" (UID: \"6822df7f-14cc-4b2c-a375-5946553b83b7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:35.192253 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.192233    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rffzj\" (UniqueName: \"kubernetes.io/projected/6822df7f-14cc-4b2c-a375-5946553b83b7-kube-api-access-rffzj\") pod \"payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv\" (UID: \"6822df7f-14cc-4b2c-a375-5946553b83b7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:35.285464 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.285418    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4975719-cc61-4442-87fe-f5e8cb580ac0-plugins-config-volume\") pod \"payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt\" (UID: \"c4975719-cc61-4442-87fe-f5e8cb580ac0\") " pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:13:35.285642 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.285496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbppv\" (UniqueName: \"kubernetes.io/projected/c4975719-cc61-4442-87fe-f5e8cb580ac0-kube-api-access-jbppv\") pod \"payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt\" (UID: \"c4975719-cc61-4442-87fe-f5e8cb580ac0\") " pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:13:35.286194 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.286173    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4975719-cc61-4442-87fe-f5e8cb580ac0-plugins-config-volume\") pod \"payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt\" (UID: \"c4975719-cc61-4442-87fe-f5e8cb580ac0\") " pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:13:35.293151 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.293122    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbppv\" (UniqueName: \"kubernetes.io/projected/c4975719-cc61-4442-87fe-f5e8cb580ac0-kube-api-access-jbppv\") pod \"payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt\" (UID: \"c4975719-cc61-4442-87fe-f5e8cb580ac0\") " pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:13:35.362249 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.362162    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:35.400657 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.400628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:13:35.514292 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.514255    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"]
Jul 30 23:13:35.519713 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:13:35.519679    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6822df7f_14cc_4b2c_a375_5946553b83b7.slice/crio-63008a3c4868050f23057c25a3c8c59777afbf357a41fe3da8ef248d4f7791d1 WatchSource:0}: Error finding container 63008a3c4868050f23057c25a3c8c59777afbf357a41fe3da8ef248d4f7791d1: Status 404 returned error can't find the container with id 63008a3c4868050f23057c25a3c8c59777afbf357a41fe3da8ef248d4f7791d1
Jul 30 23:13:35.555490 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:35.555462    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"]
Jul 30 23:13:35.558739 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:13:35.558710    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4975719_cc61_4442_87fe_f5e8cb580ac0.slice/crio-c6ffc9bc8cce77721c5b00b77fb05cecc0cfdb05fffff5dc66e5215da57ba787 WatchSource:0}: Error finding container c6ffc9bc8cce77721c5b00b77fb05cecc0cfdb05fffff5dc66e5215da57ba787: Status 404 returned error can't find the container with id c6ffc9bc8cce77721c5b00b77fb05cecc0cfdb05fffff5dc66e5215da57ba787
Jul 30 23:13:36.092671 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:36.092632    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt" event={"ID":"c4975719-cc61-4442-87fe-f5e8cb580ac0","Type":"ContainerStarted","Data":"810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe"}
Jul 30 23:13:36.093137 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:36.092705    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:13:36.093137 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:36.092731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt" event={"ID":"c4975719-cc61-4442-87fe-f5e8cb580ac0","Type":"ContainerStarted","Data":"c6ffc9bc8cce77721c5b00b77fb05cecc0cfdb05fffff5dc66e5215da57ba787"}
Jul 30 23:13:36.094169 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:36.094139    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv" event={"ID":"6822df7f-14cc-4b2c-a375-5946553b83b7","Type":"ContainerStarted","Data":"05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e"}
Jul 30 23:13:36.094259 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:36.094192    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:36.094259 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:36.094214    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv" event={"ID":"6822df7f-14cc-4b2c-a375-5946553b83b7","Type":"ContainerStarted","Data":"63008a3c4868050f23057c25a3c8c59777afbf357a41fe3da8ef248d4f7791d1"}
Jul 30 23:13:36.110978 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:36.110935    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt" podStartSLOduration=1.110921691 podStartE2EDuration="1.110921691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:13:36.109658951 +0000 UTC m=+1996.284622629" watchObservedRunningTime="2026-07-30 23:13:36.110921691 +0000 UTC m=+1996.285885388"
Jul 30 23:13:36.125455 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:36.125411    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv" podStartSLOduration=1.125397552 podStartE2EDuration="1.125397552s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:13:36.124599077 +0000 UTC m=+1996.299562746" watchObservedRunningTime="2026-07-30 23:13:36.125397552 +0000 UTC m=+1996.300361230"
Jul 30 23:13:47.100244 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:47.100213    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:13:47.100653 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:13:47.100292    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:14:16.189802 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.189755    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"]
Jul 30 23:14:16.190981 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.190038    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt" podUID="c4975719-cc61-4442-87fe-f5e8cb580ac0" containerName="payload-processing" containerID="cri-o://810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe" gracePeriod=30
Jul 30 23:14:16.196828 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.196783    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"]
Jul 30 23:14:16.197068 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.197013    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv" podUID="6822df7f-14cc-4b2c-a375-5946553b83b7" containerName="payload-pre-processing" containerID="cri-o://05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e" gracePeriod=30
Jul 30 23:14:16.463036 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.463013    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:14:16.473007 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.472984    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:14:16.542479 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.542441    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rffzj\" (UniqueName: \"kubernetes.io/projected/6822df7f-14cc-4b2c-a375-5946553b83b7-kube-api-access-rffzj\") pod \"6822df7f-14cc-4b2c-a375-5946553b83b7\" (UID: \"6822df7f-14cc-4b2c-a375-5946553b83b7\") "
Jul 30 23:14:16.542657 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.542546    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6822df7f-14cc-4b2c-a375-5946553b83b7-plugins-config-volume\") pod \"6822df7f-14cc-4b2c-a375-5946553b83b7\" (UID: \"6822df7f-14cc-4b2c-a375-5946553b83b7\") "
Jul 30 23:14:16.542657 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.542595    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbppv\" (UniqueName: \"kubernetes.io/projected/c4975719-cc61-4442-87fe-f5e8cb580ac0-kube-api-access-jbppv\") pod \"c4975719-cc61-4442-87fe-f5e8cb580ac0\" (UID: \"c4975719-cc61-4442-87fe-f5e8cb580ac0\") "
Jul 30 23:14:16.542657 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.542654    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4975719-cc61-4442-87fe-f5e8cb580ac0-plugins-config-volume\") pod \"c4975719-cc61-4442-87fe-f5e8cb580ac0\" (UID: \"c4975719-cc61-4442-87fe-f5e8cb580ac0\") "
Jul 30 23:14:16.542968 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.542938    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6822df7f-14cc-4b2c-a375-5946553b83b7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6822df7f-14cc-4b2c-a375-5946553b83b7" (UID: "6822df7f-14cc-4b2c-a375-5946553b83b7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:14:16.543270 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.543241    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4975719-cc61-4442-87fe-f5e8cb580ac0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c4975719-cc61-4442-87fe-f5e8cb580ac0" (UID: "c4975719-cc61-4442-87fe-f5e8cb580ac0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:14:16.545130 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.545096    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4975719-cc61-4442-87fe-f5e8cb580ac0-kube-api-access-jbppv" (OuterVolumeSpecName: "kube-api-access-jbppv") pod "c4975719-cc61-4442-87fe-f5e8cb580ac0" (UID: "c4975719-cc61-4442-87fe-f5e8cb580ac0"). InnerVolumeSpecName "kube-api-access-jbppv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:14:16.545214 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.545179    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6822df7f-14cc-4b2c-a375-5946553b83b7-kube-api-access-rffzj" (OuterVolumeSpecName: "kube-api-access-rffzj") pod "6822df7f-14cc-4b2c-a375-5946553b83b7" (UID: "6822df7f-14cc-4b2c-a375-5946553b83b7"). InnerVolumeSpecName "kube-api-access-rffzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:14:16.644065 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.644012    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4975719-cc61-4442-87fe-f5e8cb580ac0-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:14:16.644065 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.644067    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rffzj\" (UniqueName: \"kubernetes.io/projected/6822df7f-14cc-4b2c-a375-5946553b83b7-kube-api-access-rffzj\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:14:16.644264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.644084    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6822df7f-14cc-4b2c-a375-5946553b83b7-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:14:16.644264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:16.644107    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbppv\" (UniqueName: \"kubernetes.io/projected/c4975719-cc61-4442-87fe-f5e8cb580ac0-kube-api-access-jbppv\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:14:17.279437 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.279398    2578 generic.go:356] "Generic (PLEG): container finished" podID="6822df7f-14cc-4b2c-a375-5946553b83b7" containerID="05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e" exitCode=0
Jul 30 23:14:17.279853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.279446    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"
Jul 30 23:14:17.279853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.279473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv" event={"ID":"6822df7f-14cc-4b2c-a375-5946553b83b7","Type":"ContainerDied","Data":"05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e"}
Jul 30 23:14:17.279853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.279527    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv" event={"ID":"6822df7f-14cc-4b2c-a375-5946553b83b7","Type":"ContainerDied","Data":"63008a3c4868050f23057c25a3c8c59777afbf357a41fe3da8ef248d4f7791d1"}
Jul 30 23:14:17.279853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.279550    2578 scope.go:117] "RemoveContainer" containerID="05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e"
Jul 30 23:14:17.281036 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.281011    2578 generic.go:356] "Generic (PLEG): container finished" podID="c4975719-cc61-4442-87fe-f5e8cb580ac0" containerID="810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe" exitCode=0
Jul 30 23:14:17.281169 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.281069    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt" event={"ID":"c4975719-cc61-4442-87fe-f5e8cb580ac0","Type":"ContainerDied","Data":"810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe"}
Jul 30 23:14:17.281169 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.281082    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"
Jul 30 23:14:17.281169 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.281112    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt" event={"ID":"c4975719-cc61-4442-87fe-f5e8cb580ac0","Type":"ContainerDied","Data":"c6ffc9bc8cce77721c5b00b77fb05cecc0cfdb05fffff5dc66e5215da57ba787"}
Jul 30 23:14:17.288082 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.288042    2578 scope.go:117] "RemoveContainer" containerID="05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e"
Jul 30 23:14:17.288401 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:14:17.288378    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e\": container with ID starting with 05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e not found: ID does not exist" containerID="05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e"
Jul 30 23:14:17.288474 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.288415    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e"} err="failed to get container status \"05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e\": rpc error: code = NotFound desc = could not find container \"05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e\": container with ID starting with 05cd40b12fb36cc9e04b52d8780358e1de3db1004f59690bf1a0fb79e025669e not found: ID does not exist"
Jul 30 23:14:17.288474 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.288439    2578 scope.go:117] "RemoveContainer" containerID="810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe"
Jul 30 23:14:17.296540 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.296523    2578 scope.go:117] "RemoveContainer" containerID="810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe"
Jul 30 23:14:17.296790 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:14:17.296769    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe\": container with ID starting with 810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe not found: ID does not exist" containerID="810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe"
Jul 30 23:14:17.296845 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.296797    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe"} err="failed to get container status \"810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe\": rpc error: code = NotFound desc = could not find container \"810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe\": container with ID starting with 810b4d62a5bd4905f08cb14abf3db5e0ae94e262fe1c55917605932f78337fbe not found: ID does not exist"
Jul 30 23:14:17.304760 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.304722    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"]
Jul 30 23:14:17.306770 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.306751    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-85a1df81-667f4fc654-ktrtt"]
Jul 30 23:14:17.317280 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.317258    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"]
Jul 30 23:14:17.320757 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:17.320732    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-85a1df81-54975cd99b-ccgqv"]
Jul 30 23:14:18.342145 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:18.342104    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6822df7f-14cc-4b2c-a375-5946553b83b7" path="/var/lib/kubelet/pods/6822df7f-14cc-4b2c-a375-5946553b83b7/volumes"
Jul 30 23:14:18.342629 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:14:18.342453    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4975719-cc61-4442-87fe-f5e8cb580ac0" path="/var/lib/kubelet/pods/c4975719-cc61-4442-87fe-f5e8cb580ac0/volumes"
Jul 30 23:15:03.380119 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.380070    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"]
Jul 30 23:15:03.380641 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.380293    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6" podUID="e7d3df26-0872-4ac9-80fb-fc714bd557ee" containerName="manager" containerID="cri-o://f7005e9638b4b403250188a7db3ed32aa233433af4d716dce461d4db6df90b22" gracePeriod=10
Jul 30 23:15:03.496472 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.496443    2578 generic.go:356] "Generic (PLEG): container finished" podID="e7d3df26-0872-4ac9-80fb-fc714bd557ee" containerID="f7005e9638b4b403250188a7db3ed32aa233433af4d716dce461d4db6df90b22" exitCode=0
Jul 30 23:15:03.496575 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.496508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6" event={"ID":"e7d3df26-0872-4ac9-80fb-fc714bd557ee","Type":"ContainerDied","Data":"f7005e9638b4b403250188a7db3ed32aa233433af4d716dce461d4db6df90b22"}
Jul 30 23:15:03.640801 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.640722    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:15:03.653747 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.653714    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d3df26-0872-4ac9-80fb-fc714bd557ee-extensions-socket-volume\") pod \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\" (UID: \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\") "
Jul 30 23:15:03.653747 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.653757    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9xcl\" (UniqueName: \"kubernetes.io/projected/e7d3df26-0872-4ac9-80fb-fc714bd557ee-kube-api-access-s9xcl\") pod \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\" (UID: \"e7d3df26-0872-4ac9-80fb-fc714bd557ee\") "
Jul 30 23:15:03.654184 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.654156    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7d3df26-0872-4ac9-80fb-fc714bd557ee-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e7d3df26-0872-4ac9-80fb-fc714bd557ee" (UID: "e7d3df26-0872-4ac9-80fb-fc714bd557ee"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:15:03.656158 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.656129    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7d3df26-0872-4ac9-80fb-fc714bd557ee-kube-api-access-s9xcl" (OuterVolumeSpecName: "kube-api-access-s9xcl") pod "e7d3df26-0872-4ac9-80fb-fc714bd557ee" (UID: "e7d3df26-0872-4ac9-80fb-fc714bd557ee"). InnerVolumeSpecName "kube-api-access-s9xcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:15:03.754542 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.754502    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e7d3df26-0872-4ac9-80fb-fc714bd557ee-extensions-socket-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:15:03.754542 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:03.754538    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9xcl\" (UniqueName: \"kubernetes.io/projected/e7d3df26-0872-4ac9-80fb-fc714bd557ee-kube-api-access-s9xcl\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:15:04.502318 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:04.502289    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"
Jul 30 23:15:04.502318 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:04.502302    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6" event={"ID":"e7d3df26-0872-4ac9-80fb-fc714bd557ee","Type":"ContainerDied","Data":"9bc1be4daed2d794475cd567a46bc7e7dc1dc6c81ece785bd85026147f6cf896"}
Jul 30 23:15:04.502801 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:04.502360    2578 scope.go:117] "RemoveContainer" containerID="f7005e9638b4b403250188a7db3ed32aa233433af4d716dce461d4db6df90b22"
Jul 30 23:15:04.520487 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:04.520454    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"]
Jul 30 23:15:04.527840 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:04.527808    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-pgvc6"]
Jul 30 23:15:06.341612 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:06.341564    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7d3df26-0872-4ac9-80fb-fc714bd557ee" path="/var/lib/kubelet/pods/e7d3df26-0872-4ac9-80fb-fc714bd557ee/volumes"
Jul 30 23:15:20.368217 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:20.368189    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:15:20.372766 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:15:20.372745    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:16:19.009345 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009311    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"]
Jul 30 23:16:19.009937 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009805    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6822df7f-14cc-4b2c-a375-5946553b83b7" containerName="payload-pre-processing"
Jul 30 23:16:19.009937 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009826    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6822df7f-14cc-4b2c-a375-5946553b83b7" containerName="payload-pre-processing"
Jul 30 23:16:19.009937 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009838    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7d3df26-0872-4ac9-80fb-fc714bd557ee" containerName="manager"
Jul 30 23:16:19.009937 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009847    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7d3df26-0872-4ac9-80fb-fc714bd557ee" containerName="manager"
Jul 30 23:16:19.009937 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009881    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4975719-cc61-4442-87fe-f5e8cb580ac0" containerName="payload-processing"
Jul 30 23:16:19.009937 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009890    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4975719-cc61-4442-87fe-f5e8cb580ac0" containerName="payload-processing"
Jul 30 23:16:19.010277 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009956    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6822df7f-14cc-4b2c-a375-5946553b83b7" containerName="payload-pre-processing"
Jul 30 23:16:19.010277 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009967    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7d3df26-0872-4ac9-80fb-fc714bd557ee" containerName="manager"
Jul 30 23:16:19.010277 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.009981    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4975719-cc61-4442-87fe-f5e8cb580ac0" containerName="payload-processing"
Jul 30 23:16:19.013160 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.013133    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:19.016180 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.016160    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9q7ms\""
Jul 30 23:16:19.025096 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.025035    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"]
Jul 30 23:16:19.090150 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.090111    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bdmb\" (UniqueName: \"kubernetes.io/projected/06fd676b-515b-40a7-9205-a0623478c2ba-kube-api-access-7bdmb\") pod \"kuadrant-operator-controller-manager-68cf589589-q2hm7\" (UID: \"06fd676b-515b-40a7-9205-a0623478c2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:19.090355 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.090174    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06fd676b-515b-40a7-9205-a0623478c2ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-q2hm7\" (UID: \"06fd676b-515b-40a7-9205-a0623478c2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:19.191343 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.191306    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06fd676b-515b-40a7-9205-a0623478c2ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-q2hm7\" (UID: \"06fd676b-515b-40a7-9205-a0623478c2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:19.191521 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.191369    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bdmb\" (UniqueName: \"kubernetes.io/projected/06fd676b-515b-40a7-9205-a0623478c2ba-kube-api-access-7bdmb\") pod \"kuadrant-operator-controller-manager-68cf589589-q2hm7\" (UID: \"06fd676b-515b-40a7-9205-a0623478c2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:19.191711 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.191691    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/06fd676b-515b-40a7-9205-a0623478c2ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-q2hm7\" (UID: \"06fd676b-515b-40a7-9205-a0623478c2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:19.202882 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.202845    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bdmb\" (UniqueName: \"kubernetes.io/projected/06fd676b-515b-40a7-9205-a0623478c2ba-kube-api-access-7bdmb\") pod \"kuadrant-operator-controller-manager-68cf589589-q2hm7\" (UID: \"06fd676b-515b-40a7-9205-a0623478c2ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:19.323380 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.323280    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:19.463596 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.463551    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"]
Jul 30 23:16:19.465806 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:16:19.465780    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06fd676b_515b_40a7_9205_a0623478c2ba.slice/crio-75d67ac7f31860b283ed9149af02a5248527f5902c60b26fd59c40a6b338b260 WatchSource:0}: Error finding container 75d67ac7f31860b283ed9149af02a5248527f5902c60b26fd59c40a6b338b260: Status 404 returned error can't find the container with id 75d67ac7f31860b283ed9149af02a5248527f5902c60b26fd59c40a6b338b260
Jul 30 23:16:19.832759 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.832724    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7" event={"ID":"06fd676b-515b-40a7-9205-a0623478c2ba","Type":"ContainerStarted","Data":"c4bd05de64d11f08a018fa5bbfbe6f7b9e1f1da84ea49d2d8733fc63a125a7ee"}
Jul 30 23:16:19.832927 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.832782    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7" event={"ID":"06fd676b-515b-40a7-9205-a0623478c2ba","Type":"ContainerStarted","Data":"75d67ac7f31860b283ed9149af02a5248527f5902c60b26fd59c40a6b338b260"}
Jul 30 23:16:19.832927 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.832821    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:19.853865 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:19.853682    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7" podStartSLOduration=1.853663079 podStartE2EDuration="1.853663079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:16:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:16:19.852763226 +0000 UTC m=+2160.027726899" watchObservedRunningTime="2026-07-30 23:16:19.853663079 +0000 UTC m=+2160.028626757"
Jul 30 23:16:30.839553 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:30.839522    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-q2hm7"
Jul 30 23:16:43.034389 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.034354    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"]
Jul 30 23:16:43.037580 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.037560    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:43.039971 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.039949    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-604c6cd2\""
Jul 30 23:16:43.040101 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.040042    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-604c6cd2-dockercfg-6kknb\""
Jul 30 23:16:43.048385 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.048356    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"]
Jul 30 23:16:43.071157 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.071125    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"]
Jul 30 23:16:43.074332 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.074312    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:16:43.084641 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.084618    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"]
Jul 30 23:16:43.193160 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.193120    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m825p\" (UniqueName: \"kubernetes.io/projected/4d47c4a5-521d-4c12-a79b-a732ef25003b-kube-api-access-m825p\") pod \"payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l\" (UID: \"4d47c4a5-521d-4c12-a79b-a732ef25003b\") " pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:16:43.193342 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.193165    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z\" (UID: \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:43.193342 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.193245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvbfw\" (UniqueName: \"kubernetes.io/projected/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-kube-api-access-vvbfw\") pod \"payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z\" (UID: \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:43.193421 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.193341    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d47c4a5-521d-4c12-a79b-a732ef25003b-plugins-config-volume\") pod \"payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l\" (UID: \"4d47c4a5-521d-4c12-a79b-a732ef25003b\") " pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:16:43.294265 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.294186    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m825p\" (UniqueName: \"kubernetes.io/projected/4d47c4a5-521d-4c12-a79b-a732ef25003b-kube-api-access-m825p\") pod \"payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l\" (UID: \"4d47c4a5-521d-4c12-a79b-a732ef25003b\") " pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:16:43.294265 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.294224    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z\" (UID: \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:43.294465 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.294267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvbfw\" (UniqueName: \"kubernetes.io/projected/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-kube-api-access-vvbfw\") pod \"payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z\" (UID: \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:43.294465 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.294292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d47c4a5-521d-4c12-a79b-a732ef25003b-plugins-config-volume\") pod \"payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l\" (UID: \"4d47c4a5-521d-4c12-a79b-a732ef25003b\") " pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:16:43.294934 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.294916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z\" (UID: \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:43.294989 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.294975    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d47c4a5-521d-4c12-a79b-a732ef25003b-plugins-config-volume\") pod \"payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l\" (UID: \"4d47c4a5-521d-4c12-a79b-a732ef25003b\") " pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:16:43.302320 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.302288    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m825p\" (UniqueName: \"kubernetes.io/projected/4d47c4a5-521d-4c12-a79b-a732ef25003b-kube-api-access-m825p\") pod \"payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l\" (UID: \"4d47c4a5-521d-4c12-a79b-a732ef25003b\") " pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:16:43.302437 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.302356    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvbfw\" (UniqueName: \"kubernetes.io/projected/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-kube-api-access-vvbfw\") pod \"payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z\" (UID: \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:43.348671 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.348644    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:43.384935 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.384901    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:16:43.498686 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.498576    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"]
Jul 30 23:16:43.501411 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:16:43.501380    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19a03d4_bd85_4c74_bd05_ea06bf3cc97b.slice/crio-7289a2db03cd0c225f685a23c8b67d989145b98abfed5983715fef2482f664bc WatchSource:0}: Error finding container 7289a2db03cd0c225f685a23c8b67d989145b98abfed5983715fef2482f664bc: Status 404 returned error can't find the container with id 7289a2db03cd0c225f685a23c8b67d989145b98abfed5983715fef2482f664bc
Jul 30 23:16:43.742405 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.742165    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"]
Jul 30 23:16:43.745281 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:16:43.745245    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d47c4a5_521d_4c12_a79b_a732ef25003b.slice/crio-742aaafaad0d1e4c3460dd7a8d7545bb53457f2ff711616edab14ba4ab3ec37e WatchSource:0}: Error finding container 742aaafaad0d1e4c3460dd7a8d7545bb53457f2ff711616edab14ba4ab3ec37e: Status 404 returned error can't find the container with id 742aaafaad0d1e4c3460dd7a8d7545bb53457f2ff711616edab14ba4ab3ec37e
Jul 30 23:16:43.943548 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.943514    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z" event={"ID":"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b","Type":"ContainerStarted","Data":"bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007"}
Jul 30 23:16:43.943741 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.943577    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z" event={"ID":"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b","Type":"ContainerStarted","Data":"7289a2db03cd0c225f685a23c8b67d989145b98abfed5983715fef2482f664bc"}
Jul 30 23:16:43.943801 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.943764    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:43.945462 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.945330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l" event={"ID":"4d47c4a5-521d-4c12-a79b-a732ef25003b","Type":"ContainerStarted","Data":"4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a"}
Jul 30 23:16:43.945462 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.945375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l" event={"ID":"4d47c4a5-521d-4c12-a79b-a732ef25003b","Type":"ContainerStarted","Data":"742aaafaad0d1e4c3460dd7a8d7545bb53457f2ff711616edab14ba4ab3ec37e"}
Jul 30 23:16:43.945462 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.945410    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:16:43.959769 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.959722    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z" podStartSLOduration=0.959707407 podStartE2EDuration="959.707407ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:16:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:16:43.957679405 +0000 UTC m=+2184.132643088" watchObservedRunningTime="2026-07-30 23:16:43.959707407 +0000 UTC m=+2184.134671085"
Jul 30 23:16:43.972677 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:43.972637    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l" podStartSLOduration=0.972621776 podStartE2EDuration="972.621776ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:16:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:16:43.971258278 +0000 UTC m=+2184.146221957" watchObservedRunningTime="2026-07-30 23:16:43.972621776 +0000 UTC m=+2184.147585446"
Jul 30 23:16:54.953477 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:54.953432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:16:54.953967 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:16:54.953603    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:17:21.801985 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.801948    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"]
Jul 30 23:17:21.805745 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.805723    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:21.809430 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.809405    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-604c6cd2\"/\"openshift-service-ca.crt\""
Jul 30 23:17:21.809430 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.809424    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-604c6cd2\"/\"default-dockercfg-7x75g\""
Jul 30 23:17:21.809606 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.809404    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-604c6cd2\"/\"e2e-lifecycle-model-604c6cd2-kserve-self-signed-certs\""
Jul 30 23:17:21.809606 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.809409    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-604c6cd2\"/\"kube-root-ca.crt\""
Jul 30 23:17:21.815795 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.815769    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"]
Jul 30 23:17:21.916798 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.916760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-dshm\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:21.916983 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.916802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv7nt\" (UniqueName: \"kubernetes.io/projected/39f63823-389e-40e1-b416-cde7a0ea5985-kube-api-access-nv7nt\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:21.916983 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.916876    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-home\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:21.916983 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.916896    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f63823-389e-40e1-b416-cde7a0ea5985-tls-certs\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:21.916983 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.916949    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-tmp-dir\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:21.917140 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:21.917027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-model-cache\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.017996 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.017958    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-home\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.017996 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.017996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f63823-389e-40e1-b416-cde7a0ea5985-tls-certs\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.018241 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.018014    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-tmp-dir\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.018241 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.018091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-model-cache\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.018241 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.018110    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-dshm\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.018241 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.018125    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv7nt\" (UniqueName: \"kubernetes.io/projected/39f63823-389e-40e1-b416-cde7a0ea5985-kube-api-access-nv7nt\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.018492 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.018469    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-tmp-dir\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.018539 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.018516    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-home\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.018583 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.018562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-model-cache\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.020570 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.020549    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-dshm\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.020732 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.020715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f63823-389e-40e1-b416-cde7a0ea5985-tls-certs\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.026161 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.026137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv7nt\" (UniqueName: \"kubernetes.io/projected/39f63823-389e-40e1-b416-cde7a0ea5985-kube-api-access-nv7nt\") pod \"e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") " pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.116306 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.116211    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:22.254999 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:22.254971    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"]
Jul 30 23:17:22.257624 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:17:22.257593    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39f63823_389e_40e1_b416_cde7a0ea5985.slice/crio-273abb56dac45ca4550129166f339b1b18b698b76fba0bf6db7dbdeae28a3dd1 WatchSource:0}: Error finding container 273abb56dac45ca4550129166f339b1b18b698b76fba0bf6db7dbdeae28a3dd1: Status 404 returned error can't find the container with id 273abb56dac45ca4550129166f339b1b18b698b76fba0bf6db7dbdeae28a3dd1
Jul 30 23:17:23.130865 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:23.130826    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds" event={"ID":"39f63823-389e-40e1-b416-cde7a0ea5985","Type":"ContainerStarted","Data":"9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1"}
Jul 30 23:17:23.130865 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:23.130870    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds" event={"ID":"39f63823-389e-40e1-b416-cde7a0ea5985","Type":"ContainerStarted","Data":"273abb56dac45ca4550129166f339b1b18b698b76fba0bf6db7dbdeae28a3dd1"}
Jul 30 23:17:32.117125 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:32.117076    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:32.117612 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:32.117135    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:32.129825 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:32.129800    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:17:32.146886 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:32.146833    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds" podStartSLOduration=11.146818906 podStartE2EDuration="11.146818906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:17:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:17:23.159297425 +0000 UTC m=+2223.334261104" watchObservedRunningTime="2026-07-30 23:17:32.146818906 +0000 UTC m=+2232.321782585"
Jul 30 23:17:32.186462 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:17:32.186432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:18:06.620811 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:06.620775    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"]
Jul 30 23:18:06.621397 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:06.620979    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l" podUID="4d47c4a5-521d-4c12-a79b-a732ef25003b" containerName="payload-processing" containerID="cri-o://4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a" gracePeriod=30
Jul 30 23:18:06.632742 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:06.632712    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"]
Jul 30 23:18:06.632987 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:06.632946    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z" podUID="f19a03d4-bd85-4c74-bd05-ea06bf3cc97b" containerName="payload-pre-processing" containerID="cri-o://bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007" gracePeriod=30
Jul 30 23:18:06.914136 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:06.914107    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:18:06.917144 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:06.917121    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:18:07.009113 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.009079    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d47c4a5-521d-4c12-a79b-a732ef25003b-plugins-config-volume\") pod \"4d47c4a5-521d-4c12-a79b-a732ef25003b\" (UID: \"4d47c4a5-521d-4c12-a79b-a732ef25003b\") "
Jul 30 23:18:07.009290 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.009166    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-plugins-config-volume\") pod \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\" (UID: \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\") "
Jul 30 23:18:07.009290 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.009187    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvbfw\" (UniqueName: \"kubernetes.io/projected/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-kube-api-access-vvbfw\") pod \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\" (UID: \"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b\") "
Jul 30 23:18:07.009290 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.009211    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m825p\" (UniqueName: \"kubernetes.io/projected/4d47c4a5-521d-4c12-a79b-a732ef25003b-kube-api-access-m825p\") pod \"4d47c4a5-521d-4c12-a79b-a732ef25003b\" (UID: \"4d47c4a5-521d-4c12-a79b-a732ef25003b\") "
Jul 30 23:18:07.009511 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.009481    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d47c4a5-521d-4c12-a79b-a732ef25003b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d47c4a5-521d-4c12-a79b-a732ef25003b" (UID: "4d47c4a5-521d-4c12-a79b-a732ef25003b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:18:07.009608 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.009530    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f19a03d4-bd85-4c74-bd05-ea06bf3cc97b" (UID: "f19a03d4-bd85-4c74-bd05-ea06bf3cc97b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:18:07.011584 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.011561    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d47c4a5-521d-4c12-a79b-a732ef25003b-kube-api-access-m825p" (OuterVolumeSpecName: "kube-api-access-m825p") pod "4d47c4a5-521d-4c12-a79b-a732ef25003b" (UID: "4d47c4a5-521d-4c12-a79b-a732ef25003b"). InnerVolumeSpecName "kube-api-access-m825p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:18:07.011664 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.011594    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-kube-api-access-vvbfw" (OuterVolumeSpecName: "kube-api-access-vvbfw") pod "f19a03d4-bd85-4c74-bd05-ea06bf3cc97b" (UID: "f19a03d4-bd85-4c74-bd05-ea06bf3cc97b"). InnerVolumeSpecName "kube-api-access-vvbfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:18:07.109840 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.109799    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:07.109840 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.109835    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvbfw\" (UniqueName: \"kubernetes.io/projected/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b-kube-api-access-vvbfw\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:07.109840 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.109844    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m825p\" (UniqueName: \"kubernetes.io/projected/4d47c4a5-521d-4c12-a79b-a732ef25003b-kube-api-access-m825p\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:07.110116 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.109852    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d47c4a5-521d-4c12-a79b-a732ef25003b-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:07.339882 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.339846    2578 generic.go:356] "Generic (PLEG): container finished" podID="4d47c4a5-521d-4c12-a79b-a732ef25003b" containerID="4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a" exitCode=0
Jul 30 23:18:07.340085 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.339889    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"
Jul 30 23:18:07.340085 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.339929    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l" event={"ID":"4d47c4a5-521d-4c12-a79b-a732ef25003b","Type":"ContainerDied","Data":"4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a"}
Jul 30 23:18:07.340085 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.339980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l" event={"ID":"4d47c4a5-521d-4c12-a79b-a732ef25003b","Type":"ContainerDied","Data":"742aaafaad0d1e4c3460dd7a8d7545bb53457f2ff711616edab14ba4ab3ec37e"}
Jul 30 23:18:07.340085 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.340002    2578 scope.go:117] "RemoveContainer" containerID="4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a"
Jul 30 23:18:07.341358 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.341332    2578 generic.go:356] "Generic (PLEG): container finished" podID="f19a03d4-bd85-4c74-bd05-ea06bf3cc97b" containerID="bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007" exitCode=0
Jul 30 23:18:07.341604 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.341377    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"
Jul 30 23:18:07.341738 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.341376    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z" event={"ID":"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b","Type":"ContainerDied","Data":"bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007"}
Jul 30 23:18:07.341738 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.341717    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z" event={"ID":"f19a03d4-bd85-4c74-bd05-ea06bf3cc97b","Type":"ContainerDied","Data":"7289a2db03cd0c225f685a23c8b67d989145b98abfed5983715fef2482f664bc"}
Jul 30 23:18:07.351162 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.351039    2578 scope.go:117] "RemoveContainer" containerID="4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a"
Jul 30 23:18:07.351442 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:18:07.351418    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a\": container with ID starting with 4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a not found: ID does not exist" containerID="4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a"
Jul 30 23:18:07.351530 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.351452    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a"} err="failed to get container status \"4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a\": rpc error: code = NotFound desc = could not find container \"4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a\": container with ID starting with 4ba99ea5e437c6334d303e49a655cf49c286e25f65da3a17ce2400fa8aaedd4a not found: ID does not exist"
Jul 30 23:18:07.351530 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.351472    2578 scope.go:117] "RemoveContainer" containerID="bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007"
Jul 30 23:18:07.361880 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.361859    2578 scope.go:117] "RemoveContainer" containerID="bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007"
Jul 30 23:18:07.362275 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:18:07.362254    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007\": container with ID starting with bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007 not found: ID does not exist" containerID="bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007"
Jul 30 23:18:07.362347 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.362286    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007"} err="failed to get container status \"bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007\": rpc error: code = NotFound desc = could not find container \"bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007\": container with ID starting with bf138a50ac1d58c08de41d2dab9c2f50d72b52da01f82a4176a74b9efd852007 not found: ID does not exist"
Jul 30 23:18:07.366146 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.366124    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"]
Jul 30 23:18:07.369598 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.369579    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-604c6cd2-6f9d479f6-sm64z"]
Jul 30 23:18:07.379269 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.379235    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"]
Jul 30 23:18:07.382544 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:07.382525    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-604c6cd2-77897967bb-5z96l"]
Jul 30 23:18:08.342133 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:08.342090    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d47c4a5-521d-4c12-a79b-a732ef25003b" path="/var/lib/kubelet/pods/4d47c4a5-521d-4c12-a79b-a732ef25003b/volumes"
Jul 30 23:18:08.342619 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:08.342581    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f19a03d4-bd85-4c74-bd05-ea06bf3cc97b" path="/var/lib/kubelet/pods/f19a03d4-bd85-4c74-bd05-ea06bf3cc97b/volumes"
Jul 30 23:18:15.131119 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:15.131082    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"]
Jul 30 23:18:30.150871 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.150820    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds" podUID="39f63823-389e-40e1-b416-cde7a0ea5985" containerName="main" containerID="cri-o://9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1" gracePeriod=45
Jul 30 23:18:30.396985 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.396956    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:18:30.447509 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.447476    2578 generic.go:356] "Generic (PLEG): container finished" podID="39f63823-389e-40e1-b416-cde7a0ea5985" containerID="9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1" exitCode=0
Jul 30 23:18:30.447692 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.447541    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"
Jul 30 23:18:30.447692 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.447566    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds" event={"ID":"39f63823-389e-40e1-b416-cde7a0ea5985","Type":"ContainerDied","Data":"9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1"}
Jul 30 23:18:30.447692 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.447625    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds" event={"ID":"39f63823-389e-40e1-b416-cde7a0ea5985","Type":"ContainerDied","Data":"273abb56dac45ca4550129166f339b1b18b698b76fba0bf6db7dbdeae28a3dd1"}
Jul 30 23:18:30.447692 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.447649    2578 scope.go:117] "RemoveContainer" containerID="9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1"
Jul 30 23:18:30.456119 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.456090    2578 scope.go:117] "RemoveContainer" containerID="9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1"
Jul 30 23:18:30.456385 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:18:30.456366    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1\": container with ID starting with 9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1 not found: ID does not exist" containerID="9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1"
Jul 30 23:18:30.456451 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.456394    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1"} err="failed to get container status \"9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1\": rpc error: code = NotFound desc = could not find container \"9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1\": container with ID starting with 9561bba3c21e53f40d2108720e6c78be816c4cd6400f12d0a05e7a890895eaa1 not found: ID does not exist"
Jul 30 23:18:30.517453 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517416    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-tmp-dir\") pod \"39f63823-389e-40e1-b416-cde7a0ea5985\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") "
Jul 30 23:18:30.517627 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517463    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-dshm\") pod \"39f63823-389e-40e1-b416-cde7a0ea5985\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") "
Jul 30 23:18:30.517627 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517498    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-model-cache\") pod \"39f63823-389e-40e1-b416-cde7a0ea5985\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") "
Jul 30 23:18:30.517627 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517521    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-home\") pod \"39f63823-389e-40e1-b416-cde7a0ea5985\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") "
Jul 30 23:18:30.517627 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517562    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nv7nt\" (UniqueName: \"kubernetes.io/projected/39f63823-389e-40e1-b416-cde7a0ea5985-kube-api-access-nv7nt\") pod \"39f63823-389e-40e1-b416-cde7a0ea5985\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") "
Jul 30 23:18:30.517627 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517583    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f63823-389e-40e1-b416-cde7a0ea5985-tls-certs\") pod \"39f63823-389e-40e1-b416-cde7a0ea5985\" (UID: \"39f63823-389e-40e1-b416-cde7a0ea5985\") "
Jul 30 23:18:30.517889 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517689    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "39f63823-389e-40e1-b416-cde7a0ea5985" (UID: "39f63823-389e-40e1-b416-cde7a0ea5985"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:18:30.517889 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517793    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-home" (OuterVolumeSpecName: "home") pod "39f63823-389e-40e1-b416-cde7a0ea5985" (UID: "39f63823-389e-40e1-b416-cde7a0ea5985"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:18:30.517889 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517810    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-model-cache" (OuterVolumeSpecName: "model-cache") pod "39f63823-389e-40e1-b416-cde7a0ea5985" (UID: "39f63823-389e-40e1-b416-cde7a0ea5985"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:18:30.517889 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.517822    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:30.519881 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.519855    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39f63823-389e-40e1-b416-cde7a0ea5985-kube-api-access-nv7nt" (OuterVolumeSpecName: "kube-api-access-nv7nt") pod "39f63823-389e-40e1-b416-cde7a0ea5985" (UID: "39f63823-389e-40e1-b416-cde7a0ea5985"). InnerVolumeSpecName "kube-api-access-nv7nt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:18:30.520002 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.519947    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-dshm" (OuterVolumeSpecName: "dshm") pod "39f63823-389e-40e1-b416-cde7a0ea5985" (UID: "39f63823-389e-40e1-b416-cde7a0ea5985"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:18:30.520002 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.519956    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39f63823-389e-40e1-b416-cde7a0ea5985-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "39f63823-389e-40e1-b416-cde7a0ea5985" (UID: "39f63823-389e-40e1-b416-cde7a0ea5985"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:18:30.618893 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.618852    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:30.618893 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.618883    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:30.618893 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.618892    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nv7nt\" (UniqueName: \"kubernetes.io/projected/39f63823-389e-40e1-b416-cde7a0ea5985-kube-api-access-nv7nt\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:30.618893 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.618902    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39f63823-389e-40e1-b416-cde7a0ea5985-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:30.619203 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.618910    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39f63823-389e-40e1-b416-cde7a0ea5985-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:18:30.768800 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.768771    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"]
Jul 30 23:18:30.771567 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:30.771543    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-604c6cd2/e2e-lifecycle-model-604c6cd2-kserve-55ccb4bc95-hrmds"]
Jul 30 23:18:32.341630 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:18:32.341598    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39f63823-389e-40e1-b416-cde7a0ea5985" path="/var/lib/kubelet/pods/39f63823-389e-40e1-b416-cde7a0ea5985/volumes"
Jul 30 23:20:09.278655 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.278618    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"]
Jul 30 23:20:09.279175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.278951    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39f63823-389e-40e1-b416-cde7a0ea5985" containerName="main"
Jul 30 23:20:09.279175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.278962    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="39f63823-389e-40e1-b416-cde7a0ea5985" containerName="main"
Jul 30 23:20:09.279175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.278974    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f19a03d4-bd85-4c74-bd05-ea06bf3cc97b" containerName="payload-pre-processing"
Jul 30 23:20:09.279175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.278979    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f19a03d4-bd85-4c74-bd05-ea06bf3cc97b" containerName="payload-pre-processing"
Jul 30 23:20:09.279175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.278991    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d47c4a5-521d-4c12-a79b-a732ef25003b" containerName="payload-processing"
Jul 30 23:20:09.279175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.278997    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d47c4a5-521d-4c12-a79b-a732ef25003b" containerName="payload-processing"
Jul 30 23:20:09.279175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.279064    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f19a03d4-bd85-4c74-bd05-ea06bf3cc97b" containerName="payload-pre-processing"
Jul 30 23:20:09.279175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.279070    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d47c4a5-521d-4c12-a79b-a732ef25003b" containerName="payload-processing"
Jul 30 23:20:09.279175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.279079    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="39f63823-389e-40e1-b416-cde7a0ea5985" containerName="main"
Jul 30 23:20:09.281735 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.281719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:09.284453 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.284425    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-fe64f5-dockercfg-cjmkc\""
Jul 30 23:20:09.284594 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.284514    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-fe64f5\""
Jul 30 23:20:09.291914 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.291888    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"]
Jul 30 23:20:09.316272 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.316239    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"]
Jul 30 23:20:09.319429 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.319409    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:09.329566 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.329539    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"]
Jul 30 23:20:09.373574 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.373540    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwqf4\" (UniqueName: \"kubernetes.io/projected/4aa15b97-c2fa-49ce-9601-6763d5b3d367-kube-api-access-zwqf4\") pod \"payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84\" (UID: \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:09.373574 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.373585    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klmsg\" (UniqueName: \"kubernetes.io/projected/69ae2e23-1846-439c-b9a7-3a4d0af360a1-kube-api-access-klmsg\") pod \"payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d\" (UID: \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:09.373806 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.373603    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa15b97-c2fa-49ce-9601-6763d5b3d367-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84\" (UID: \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:09.373806 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.373709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ae2e23-1846-439c-b9a7-3a4d0af360a1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d\" (UID: \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:09.474533 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.474479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwqf4\" (UniqueName: \"kubernetes.io/projected/4aa15b97-c2fa-49ce-9601-6763d5b3d367-kube-api-access-zwqf4\") pod \"payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84\" (UID: \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:09.474533 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.474541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klmsg\" (UniqueName: \"kubernetes.io/projected/69ae2e23-1846-439c-b9a7-3a4d0af360a1-kube-api-access-klmsg\") pod \"payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d\" (UID: \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:09.474800 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.474784    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa15b97-c2fa-49ce-9601-6763d5b3d367-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84\" (UID: \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:09.474887 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.474873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ae2e23-1846-439c-b9a7-3a4d0af360a1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d\" (UID: \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:09.475559 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.475531    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa15b97-c2fa-49ce-9601-6763d5b3d367-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84\" (UID: \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:09.475672 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.475541    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ae2e23-1846-439c-b9a7-3a4d0af360a1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d\" (UID: \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:09.482860 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.482832    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwqf4\" (UniqueName: \"kubernetes.io/projected/4aa15b97-c2fa-49ce-9601-6763d5b3d367-kube-api-access-zwqf4\") pod \"payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84\" (UID: \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:09.483035 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.483010    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klmsg\" (UniqueName: \"kubernetes.io/projected/69ae2e23-1846-439c-b9a7-3a4d0af360a1-kube-api-access-klmsg\") pod \"payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d\" (UID: \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:09.592507 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.592416    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:09.629471 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.629429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:09.746390 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.745848    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"]
Jul 30 23:20:09.747689 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:20:09.747652    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa15b97_c2fa_49ce_9601_6763d5b3d367.slice/crio-817daf1f09694057f576041be4064ea6ec9f7267d053fbc10e4fb0ba3f1d4355 WatchSource:0}: Error finding container 817daf1f09694057f576041be4064ea6ec9f7267d053fbc10e4fb0ba3f1d4355: Status 404 returned error can't find the container with id 817daf1f09694057f576041be4064ea6ec9f7267d053fbc10e4fb0ba3f1d4355
Jul 30 23:20:09.795024 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.794999    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"]
Jul 30 23:20:09.797184 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:20:09.797155    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69ae2e23_1846_439c_b9a7_3a4d0af360a1.slice/crio-683d1cb55c7dbe6856431bb20a829ce6117132ae5b0a69424364d71f2fc38dd7 WatchSource:0}: Error finding container 683d1cb55c7dbe6856431bb20a829ce6117132ae5b0a69424364d71f2fc38dd7: Status 404 returned error can't find the container with id 683d1cb55c7dbe6856431bb20a829ce6117132ae5b0a69424364d71f2fc38dd7
Jul 30 23:20:09.889694 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.889658    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d" event={"ID":"69ae2e23-1846-439c-b9a7-3a4d0af360a1","Type":"ContainerStarted","Data":"6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc"}
Jul 30 23:20:09.889867 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.889720    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d" event={"ID":"69ae2e23-1846-439c-b9a7-3a4d0af360a1","Type":"ContainerStarted","Data":"683d1cb55c7dbe6856431bb20a829ce6117132ae5b0a69424364d71f2fc38dd7"}
Jul 30 23:20:09.889867 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.889812    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:09.891199 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.891159    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84" event={"ID":"4aa15b97-c2fa-49ce-9601-6763d5b3d367","Type":"ContainerStarted","Data":"bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1"}
Jul 30 23:20:09.891341 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.891217    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84" event={"ID":"4aa15b97-c2fa-49ce-9601-6763d5b3d367","Type":"ContainerStarted","Data":"817daf1f09694057f576041be4064ea6ec9f7267d053fbc10e4fb0ba3f1d4355"}
Jul 30 23:20:09.891341 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.891246    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:09.906788 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.906733    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d" podStartSLOduration=0.906716832 podStartE2EDuration="906.716832ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:20:09.905982463 +0000 UTC m=+2390.080946143" watchObservedRunningTime="2026-07-30 23:20:09.906716832 +0000 UTC m=+2390.081680511"
Jul 30 23:20:09.922786 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:09.922734    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84" podStartSLOduration=0.922719432 podStartE2EDuration="922.719432ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:20:09.921950067 +0000 UTC m=+2390.096913748" watchObservedRunningTime="2026-07-30 23:20:09.922719432 +0000 UTC m=+2390.097683110"
Jul 30 23:20:20.392945 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:20.392917    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:20:20.398502 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:20.398480    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:20:20.898804 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:20.898747    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84" podUID="4aa15b97-c2fa-49ce-9601-6763d5b3d367" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.55:9004: connect: connection refused"
Jul 30 23:20:20.899153 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:20.899135    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:20:30.897509 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:30.897476    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:20:38.317777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.317740    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"]
Jul 30 23:20:38.321125 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.321108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:38.324977 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.324948    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-04f0cc\""
Jul 30 23:20:38.325121 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.325042    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-04f0cc-dockercfg-4j9pp\""
Jul 30 23:20:38.334964 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.334934    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"]
Jul 30 23:20:38.350518 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.350487    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"]
Jul 30 23:20:38.354313 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.354288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:20:38.361596 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.361297    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"]
Jul 30 23:20:38.438457 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.438423    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40363491-9e8e-4b27-b8c9-cfec9b74b545-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k\" (UID: \"40363491-9e8e-4b27-b8c9-cfec9b74b545\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:38.438629 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.438495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww\" (UID: \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:20:38.438629 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.438538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77lj8\" (UniqueName: \"kubernetes.io/projected/40363491-9e8e-4b27-b8c9-cfec9b74b545-kube-api-access-77lj8\") pod \"payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k\" (UID: \"40363491-9e8e-4b27-b8c9-cfec9b74b545\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:38.438629 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.438556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpl9m\" (UniqueName: \"kubernetes.io/projected/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-kube-api-access-cpl9m\") pod \"payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww\" (UID: \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:20:38.539734 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.539693    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77lj8\" (UniqueName: \"kubernetes.io/projected/40363491-9e8e-4b27-b8c9-cfec9b74b545-kube-api-access-77lj8\") pod \"payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k\" (UID: \"40363491-9e8e-4b27-b8c9-cfec9b74b545\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:38.539734 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.539732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpl9m\" (UniqueName: \"kubernetes.io/projected/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-kube-api-access-cpl9m\") pod \"payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww\" (UID: \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:20:38.540007 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.539765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40363491-9e8e-4b27-b8c9-cfec9b74b545-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k\" (UID: \"40363491-9e8e-4b27-b8c9-cfec9b74b545\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:38.540007 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.539808    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww\" (UID: \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:20:38.540492 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.540473    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww\" (UID: \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:20:38.540546 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.540505    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40363491-9e8e-4b27-b8c9-cfec9b74b545-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k\" (UID: \"40363491-9e8e-4b27-b8c9-cfec9b74b545\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:38.554602 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.554571    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpl9m\" (UniqueName: \"kubernetes.io/projected/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-kube-api-access-cpl9m\") pod \"payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww\" (UID: \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:20:38.554784 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.554761    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77lj8\" (UniqueName: \"kubernetes.io/projected/40363491-9e8e-4b27-b8c9-cfec9b74b545-kube-api-access-77lj8\") pod \"payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k\" (UID: \"40363491-9e8e-4b27-b8c9-cfec9b74b545\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:38.631302 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.631207    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:38.665453 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.665424    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:20:38.779910 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.779858    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"]
Jul 30 23:20:38.781166 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:20:38.781136    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40363491_9e8e_4b27_b8c9_cfec9b74b545.slice/crio-51bf9f51531929cda55be0ac0f7aab5e29d322ab3350f6459484dddf9d3044b2 WatchSource:0}: Error finding container 51bf9f51531929cda55be0ac0f7aab5e29d322ab3350f6459484dddf9d3044b2: Status 404 returned error can't find the container with id 51bf9f51531929cda55be0ac0f7aab5e29d322ab3350f6459484dddf9d3044b2
Jul 30 23:20:38.822425 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:38.822399    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"]
Jul 30 23:20:38.824539 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:20:38.824517    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99ec8eac_4cbb_4161_a86d_c5714b8ed4ae.slice/crio-d7c41470a77ee705d0bbcd848d825389c4b4c303e1c23b0b0e6c8e4135003fb3 WatchSource:0}: Error finding container d7c41470a77ee705d0bbcd848d825389c4b4c303e1c23b0b0e6c8e4135003fb3: Status 404 returned error can't find the container with id d7c41470a77ee705d0bbcd848d825389c4b4c303e1c23b0b0e6c8e4135003fb3
Jul 30 23:20:39.023891 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:39.023843    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww" event={"ID":"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae","Type":"ContainerStarted","Data":"892efb9cc800a663e1219cf2df192584e1f623465c7a868bccf47a563a04a47b"}
Jul 30 23:20:39.024090 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:39.023909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww" event={"ID":"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae","Type":"ContainerStarted","Data":"d7c41470a77ee705d0bbcd848d825389c4b4c303e1c23b0b0e6c8e4135003fb3"}
Jul 30 23:20:39.024090 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:39.023942    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:20:39.025380 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:39.025351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k" event={"ID":"40363491-9e8e-4b27-b8c9-cfec9b74b545","Type":"ContainerStarted","Data":"697a89f1fb4d2f151759a8127fdcc0b7c62a3a72502c0367eb71367c3b86445c"}
Jul 30 23:20:39.025492 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:39.025397    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:39.025492 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:39.025412    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k" event={"ID":"40363491-9e8e-4b27-b8c9-cfec9b74b545","Type":"ContainerStarted","Data":"51bf9f51531929cda55be0ac0f7aab5e29d322ab3350f6459484dddf9d3044b2"}
Jul 30 23:20:39.041531 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:39.041480    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww" podStartSLOduration=1.041465336 podStartE2EDuration="1.041465336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:20:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:20:39.040610396 +0000 UTC m=+2419.215574075" watchObservedRunningTime="2026-07-30 23:20:39.041465336 +0000 UTC m=+2419.216429014"
Jul 30 23:20:39.060303 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:39.060244    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k" podStartSLOduration=1.060222309 podStartE2EDuration="1.060222309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:20:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:20:39.057255963 +0000 UTC m=+2419.232219645" watchObservedRunningTime="2026-07-30 23:20:39.060222309 +0000 UTC m=+2419.235185991"
Jul 30 23:20:50.031911 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:50.031879    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:20:50.032414 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:20:50.031967    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:21:15.800252 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.800221    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"]
Jul 30 23:21:15.803894 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.803875    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.807081 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.807039    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fe64f5\"/\"kube-root-ca.crt\""
Jul 30 23:21:15.808122 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.808098    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-fe64f5\"/\"openshift-service-ca.crt\""
Jul 30 23:21:15.808122 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.808117    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fe64f5\"/\"default-dockercfg-ztwf4\""
Jul 30 23:21:15.808250 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.808099    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-fe64f5\"/\"test-model-fe64f5-kserve-self-signed-certs\""
Jul 30 23:21:15.813400 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.813374    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"]
Jul 30 23:21:15.862296 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.862266    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4xzc\" (UniqueName: \"kubernetes.io/projected/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-kube-api-access-h4xzc\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.862456 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.862313    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tmp-dir\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.862456 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.862364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tls-certs\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.862456 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.862446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-model-cache\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.862560 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.862467    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-home\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.862560 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.862489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-dshm\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.963729 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.963680    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-model-cache\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.963936 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.963747    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-home\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.963936 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.963788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-dshm\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.963936 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.963853    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4xzc\" (UniqueName: \"kubernetes.io/projected/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-kube-api-access-h4xzc\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.963936 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.963923    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tmp-dir\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.964186 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.963953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tls-certs\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.969652 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.969614    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-home\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.969849 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.969656    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-model-cache\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.969849 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.969676    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tmp-dir\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.970185 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.970079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tls-certs\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.971518 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.971495    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-dshm\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:15.978709 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:15.978680    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4xzc\" (UniqueName: \"kubernetes.io/projected/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-kube-api-access-h4xzc\") pod \"test-model-fe64f5-kserve-59fcf8c9b9-ttq85\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") " pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:16.114877 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:16.114782    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:16.250642 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:16.250617    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"]
Jul 30 23:21:16.252981 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:21:16.252958    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fc05dd8_fbf4_4b46_8e44_5c9f778ce204.slice/crio-47647037e0b817bee7351556cc358437cfad2bd9e5a087f6b8288bca556d96ec WatchSource:0}: Error finding container 47647037e0b817bee7351556cc358437cfad2bd9e5a087f6b8288bca556d96ec: Status 404 returned error can't find the container with id 47647037e0b817bee7351556cc358437cfad2bd9e5a087f6b8288bca556d96ec
Jul 30 23:21:17.203601 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:17.203563    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85" event={"ID":"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204","Type":"ContainerStarted","Data":"550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7"}
Jul 30 23:21:17.203601 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:17.203607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85" event={"ID":"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204","Type":"ContainerStarted","Data":"47647037e0b817bee7351556cc358437cfad2bd9e5a087f6b8288bca556d96ec"}
Jul 30 23:21:17.221475 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:17.221424    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85" podStartSLOduration=2.2214071 podStartE2EDuration="2.2214071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:21:17.220537803 +0000 UTC m=+2457.395501483" watchObservedRunningTime="2026-07-30 23:21:17.2214071 +0000 UTC m=+2457.396370815"
Jul 30 23:21:26.115315 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:26.115263    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:26.115866 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:26.115336    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:26.128207 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:26.128165    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:21:26.257654 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:21:26.257615    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:22:00.005161 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.005124    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"]
Jul 30 23:22:00.008411 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.008392    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.011045 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.011020    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-04f0cc\"/\"openshift-service-ca.crt\""
Jul 30 23:22:00.011175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.011039    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-04f0cc\"/\"kube-root-ca.crt\""
Jul 30 23:22:00.012408 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.012391    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-04f0cc\"/\"test-model-04f0cc-kserve-self-signed-certs\""
Jul 30 23:22:00.012508 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.012396    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-04f0cc\"/\"default-dockercfg-j9fp6\""
Jul 30 23:22:00.017525 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.017501    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"]
Jul 30 23:22:00.147753 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.147719    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tls-certs\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.147753 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.147756    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgdjh\" (UniqueName: \"kubernetes.io/projected/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-kube-api-access-hgdjh\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.147996 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.147793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-home\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.147996 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.147928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-dshm\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.148129 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.147998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-model-cache\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.148129 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.148040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tmp-dir\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.249454 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.249412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-home\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.249633 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.249483    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-dshm\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.249633 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.249537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-model-cache\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.249633 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.249575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tmp-dir\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.249633 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.249603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tls-certs\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.249633 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.249627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgdjh\" (UniqueName: \"kubernetes.io/projected/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-kube-api-access-hgdjh\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.249896 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.249873    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-home\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.249965 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.249946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-model-cache\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.250041 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.250017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tmp-dir\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.251984 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.251964    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-dshm\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.252199 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.252179    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tls-certs\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.257384 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.257332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgdjh\" (UniqueName: \"kubernetes.io/projected/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-kube-api-access-hgdjh\") pod \"test-model-04f0cc-kserve-6677b9f5c-rnpcn\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") " pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.318786 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.318746    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:00.459632 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:00.459503    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"]
Jul 30 23:22:00.462177 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:22:00.462147    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a5a8c7e_bbcc_448f_8eb4_5a302388853d.slice/crio-97cb2f2c605cc1cd38aa1399f033dd7b3ce71dae1924aef2cea1f822e5f07bb1 WatchSource:0}: Error finding container 97cb2f2c605cc1cd38aa1399f033dd7b3ce71dae1924aef2cea1f822e5f07bb1: Status 404 returned error can't find the container with id 97cb2f2c605cc1cd38aa1399f033dd7b3ce71dae1924aef2cea1f822e5f07bb1
Jul 30 23:22:01.421766 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:01.421727    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn" event={"ID":"7a5a8c7e-bbcc-448f-8eb4-5a302388853d","Type":"ContainerStarted","Data":"ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f"}
Jul 30 23:22:01.422191 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:01.421778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn" event={"ID":"7a5a8c7e-bbcc-448f-8eb4-5a302388853d","Type":"ContainerStarted","Data":"97cb2f2c605cc1cd38aa1399f033dd7b3ce71dae1924aef2cea1f822e5f07bb1"}
Jul 30 23:22:01.440232 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:01.440176    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn" podStartSLOduration=2.440158264 podStartE2EDuration="2.440158264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:21:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:22:01.439749832 +0000 UTC m=+2501.614713512" watchObservedRunningTime="2026-07-30 23:22:01.440158264 +0000 UTC m=+2501.615121985"
Jul 30 23:22:10.319391 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:10.319344    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:10.319847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:10.319404    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:10.332440 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:10.332407    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:10.478949 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:10.478920    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:42.125344 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:42.125304    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"]
Jul 30 23:22:43.024559 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:43.024524    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"]
Jul 30 23:22:53.809384 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:53.809344    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"]
Jul 30 23:22:53.810155 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:53.809624    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d" podUID="69ae2e23-1846-439c-b9a7-3a4d0af360a1" containerName="payload-processing" containerID="cri-o://6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc" gracePeriod=30
Jul 30 23:22:53.813777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:53.813748    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"]
Jul 30 23:22:53.814208 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:53.814178    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84" podUID="4aa15b97-c2fa-49ce-9601-6763d5b3d367" containerName="payload-pre-processing" containerID="cri-o://bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1" gracePeriod=30
Jul 30 23:22:54.086110 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.086085    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:22:54.100489 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.100468    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:22:54.226943 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.226912    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwqf4\" (UniqueName: \"kubernetes.io/projected/4aa15b97-c2fa-49ce-9601-6763d5b3d367-kube-api-access-zwqf4\") pod \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\" (UID: \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\") "
Jul 30 23:22:54.226943 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.226949    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klmsg\" (UniqueName: \"kubernetes.io/projected/69ae2e23-1846-439c-b9a7-3a4d0af360a1-kube-api-access-klmsg\") pod \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\" (UID: \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\") "
Jul 30 23:22:54.227253 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.226968    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ae2e23-1846-439c-b9a7-3a4d0af360a1-plugins-config-volume\") pod \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\" (UID: \"69ae2e23-1846-439c-b9a7-3a4d0af360a1\") "
Jul 30 23:22:54.227253 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.227091    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa15b97-c2fa-49ce-9601-6763d5b3d367-plugins-config-volume\") pod \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\" (UID: \"4aa15b97-c2fa-49ce-9601-6763d5b3d367\") "
Jul 30 23:22:54.227363 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.227323    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69ae2e23-1846-439c-b9a7-3a4d0af360a1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "69ae2e23-1846-439c-b9a7-3a4d0af360a1" (UID: "69ae2e23-1846-439c-b9a7-3a4d0af360a1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:22:54.227486 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.227460    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aa15b97-c2fa-49ce-9601-6763d5b3d367-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4aa15b97-c2fa-49ce-9601-6763d5b3d367" (UID: "4aa15b97-c2fa-49ce-9601-6763d5b3d367"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:22:54.229276 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.229254    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa15b97-c2fa-49ce-9601-6763d5b3d367-kube-api-access-zwqf4" (OuterVolumeSpecName: "kube-api-access-zwqf4") pod "4aa15b97-c2fa-49ce-9601-6763d5b3d367" (UID: "4aa15b97-c2fa-49ce-9601-6763d5b3d367"). InnerVolumeSpecName "kube-api-access-zwqf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:22:54.229366 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.229344    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69ae2e23-1846-439c-b9a7-3a4d0af360a1-kube-api-access-klmsg" (OuterVolumeSpecName: "kube-api-access-klmsg") pod "69ae2e23-1846-439c-b9a7-3a4d0af360a1" (UID: "69ae2e23-1846-439c-b9a7-3a4d0af360a1"). InnerVolumeSpecName "kube-api-access-klmsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:22:54.327761 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.327678    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4aa15b97-c2fa-49ce-9601-6763d5b3d367-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:54.327761 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.327704    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwqf4\" (UniqueName: \"kubernetes.io/projected/4aa15b97-c2fa-49ce-9601-6763d5b3d367-kube-api-access-zwqf4\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:54.327761 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.327716    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klmsg\" (UniqueName: \"kubernetes.io/projected/69ae2e23-1846-439c-b9a7-3a4d0af360a1-kube-api-access-klmsg\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:54.327761 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.327725    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/69ae2e23-1846-439c-b9a7-3a4d0af360a1-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:54.676836 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.676754    2578 generic.go:356] "Generic (PLEG): container finished" podID="69ae2e23-1846-439c-b9a7-3a4d0af360a1" containerID="6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc" exitCode=0
Jul 30 23:22:54.676836 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.676800    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"
Jul 30 23:22:54.677021 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.676836    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d" event={"ID":"69ae2e23-1846-439c-b9a7-3a4d0af360a1","Type":"ContainerDied","Data":"6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc"}
Jul 30 23:22:54.677021 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.676884    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d" event={"ID":"69ae2e23-1846-439c-b9a7-3a4d0af360a1","Type":"ContainerDied","Data":"683d1cb55c7dbe6856431bb20a829ce6117132ae5b0a69424364d71f2fc38dd7"}
Jul 30 23:22:54.677021 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.676913    2578 scope.go:117] "RemoveContainer" containerID="6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc"
Jul 30 23:22:54.678174 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.678153    2578 generic.go:356] "Generic (PLEG): container finished" podID="4aa15b97-c2fa-49ce-9601-6763d5b3d367" containerID="bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1" exitCode=0
Jul 30 23:22:54.678269 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.678182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84" event={"ID":"4aa15b97-c2fa-49ce-9601-6763d5b3d367","Type":"ContainerDied","Data":"bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1"}
Jul 30 23:22:54.678269 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.678202    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"
Jul 30 23:22:54.678269 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.678240    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84" event={"ID":"4aa15b97-c2fa-49ce-9601-6763d5b3d367","Type":"ContainerDied","Data":"817daf1f09694057f576041be4064ea6ec9f7267d053fbc10e4fb0ba3f1d4355"}
Jul 30 23:22:54.686159 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.686131    2578 scope.go:117] "RemoveContainer" containerID="6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc"
Jul 30 23:22:54.686452 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:22:54.686431    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc\": container with ID starting with 6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc not found: ID does not exist" containerID="6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc"
Jul 30 23:22:54.686544 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.686460    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc"} err="failed to get container status \"6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc\": rpc error: code = NotFound desc = could not find container \"6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc\": container with ID starting with 6a771ed88f7d9cf670614366612c66216c59eba0faef4844c23d2a5c54b9cfcc not found: ID does not exist"
Jul 30 23:22:54.686544 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.686476    2578 scope.go:117] "RemoveContainer" containerID="bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1"
Jul 30 23:22:54.695445 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.695416    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"]
Jul 30 23:22:54.695684 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.695664    2578 scope.go:117] "RemoveContainer" containerID="bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1"
Jul 30 23:22:54.696475 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:22:54.696434    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1\": container with ID starting with bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1 not found: ID does not exist" containerID="bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1"
Jul 30 23:22:54.696595 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.696493    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1"} err="failed to get container status \"bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1\": rpc error: code = NotFound desc = could not find container \"bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1\": container with ID starting with bc8c44dbd91eff77cb779b909112dab6ee0ce0c1b7e745224a1e8893eb7debf1 not found: ID does not exist"
Jul 30 23:22:54.700890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.700860    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-fe64f5-65957d9cbb-2ql4d"]
Jul 30 23:22:54.710967 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.710940    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"]
Jul 30 23:22:54.715223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:54.715200    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-fe64f5-68b6c79dfb-slr84"]
Jul 30 23:22:56.343225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:56.343188    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4aa15b97-c2fa-49ce-9601-6763d5b3d367" path="/var/lib/kubelet/pods/4aa15b97-c2fa-49ce-9601-6763d5b3d367/volumes"
Jul 30 23:22:56.343583 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:56.343543    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69ae2e23-1846-439c-b9a7-3a4d0af360a1" path="/var/lib/kubelet/pods/69ae2e23-1846-439c-b9a7-3a4d0af360a1/volumes"
Jul 30 23:22:57.146386 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.146346    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85" podUID="8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" containerName="main" containerID="cri-o://550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7" gracePeriod=45
Jul 30 23:22:57.397863 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.397796    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:22:57.558579 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.558541    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-model-cache\") pod \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") "
Jul 30 23:22:57.558579 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.558578    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-home\") pod \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") "
Jul 30 23:22:57.558838 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.558622    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4xzc\" (UniqueName: \"kubernetes.io/projected/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-kube-api-access-h4xzc\") pod \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") "
Jul 30 23:22:57.558838 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.558641    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-dshm\") pod \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") "
Jul 30 23:22:57.558838 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.558677    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tls-certs\") pod \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") "
Jul 30 23:22:57.558838 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.558703    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tmp-dir\") pod \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\" (UID: \"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204\") "
Jul 30 23:22:57.559042 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.558893    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-home" (OuterVolumeSpecName: "home") pod "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" (UID: "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:22:57.559042 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.558907    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-model-cache" (OuterVolumeSpecName: "model-cache") pod "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" (UID: "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:22:57.559242 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.559135    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:57.559242 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.559155    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:57.559242 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.559171    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" (UID: "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:22:57.561138 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.561111    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-dshm" (OuterVolumeSpecName: "dshm") pod "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" (UID: "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:22:57.561269 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.561145    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-kube-api-access-h4xzc" (OuterVolumeSpecName: "kube-api-access-h4xzc") pod "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" (UID: "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204"). InnerVolumeSpecName "kube-api-access-h4xzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:22:57.561269 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.561157    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" (UID: "8fc05dd8-fbf4-4b46-8e44-5c9f778ce204"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:22:57.660440 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.660346    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4xzc\" (UniqueName: \"kubernetes.io/projected/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-kube-api-access-h4xzc\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:57.660440 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.660391    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:57.660634 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.660453    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:57.660634 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.660469    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:57.694267 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.694231    2578 generic.go:356] "Generic (PLEG): container finished" podID="8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" containerID="550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7" exitCode=0
Jul 30 23:22:57.694454 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.694276    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"
Jul 30 23:22:57.694454 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.694328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85" event={"ID":"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204","Type":"ContainerDied","Data":"550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7"}
Jul 30 23:22:57.694454 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.694369    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85" event={"ID":"8fc05dd8-fbf4-4b46-8e44-5c9f778ce204","Type":"ContainerDied","Data":"47647037e0b817bee7351556cc358437cfad2bd9e5a087f6b8288bca556d96ec"}
Jul 30 23:22:57.694454 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.694391    2578 scope.go:117] "RemoveContainer" containerID="550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7"
Jul 30 23:22:57.703503 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.703485    2578 scope.go:117] "RemoveContainer" containerID="550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7"
Jul 30 23:22:57.703771 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:22:57.703750    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7\": container with ID starting with 550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7 not found: ID does not exist" containerID="550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7"
Jul 30 23:22:57.703821 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.703781    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7"} err="failed to get container status \"550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7\": rpc error: code = NotFound desc = could not find container \"550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7\": container with ID starting with 550c081af3df7566eec65f8862f32d30b3533bdd261bf52095434b42c9e370c7 not found: ID does not exist"
Jul 30 23:22:57.715989 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.715960    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"]
Jul 30 23:22:57.719615 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:57.719591    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-fe64f5/test-model-fe64f5-kserve-59fcf8c9b9-ttq85"]
Jul 30 23:22:58.046505 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.046457    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn" podUID="7a5a8c7e-bbcc-448f-8eb4-5a302388853d" containerName="main" containerID="cri-o://ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f" gracePeriod=45
Jul 30 23:22:58.299173 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.299104    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:58.344025 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.343991    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" path="/var/lib/kubelet/pods/8fc05dd8-fbf4-4b46-8e44-5c9f778ce204/volumes"
Jul 30 23:22:58.468755 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.468722    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tls-certs\") pod \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") "
Jul 30 23:22:58.469225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.468773    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tmp-dir\") pod \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") "
Jul 30 23:22:58.469225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.468802    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-dshm\") pod \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") "
Jul 30 23:22:58.469225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.468821    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-model-cache\") pod \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") "
Jul 30 23:22:58.469225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.468947    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-home\") pod \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") "
Jul 30 23:22:58.469225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.469041    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgdjh\" (UniqueName: \"kubernetes.io/projected/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-kube-api-access-hgdjh\") pod \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\" (UID: \"7a5a8c7e-bbcc-448f-8eb4-5a302388853d\") "
Jul 30 23:22:58.469225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.469089    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-model-cache" (OuterVolumeSpecName: "model-cache") pod "7a5a8c7e-bbcc-448f-8eb4-5a302388853d" (UID: "7a5a8c7e-bbcc-448f-8eb4-5a302388853d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:22:58.469225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.469109    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7a5a8c7e-bbcc-448f-8eb4-5a302388853d" (UID: "7a5a8c7e-bbcc-448f-8eb4-5a302388853d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:22:58.469225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.469207    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-home" (OuterVolumeSpecName: "home") pod "7a5a8c7e-bbcc-448f-8eb4-5a302388853d" (UID: "7a5a8c7e-bbcc-448f-8eb4-5a302388853d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:22:58.469574 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.469330    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:58.469574 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.469344    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:58.469574 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.469352    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:58.471223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.471193    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7a5a8c7e-bbcc-448f-8eb4-5a302388853d" (UID: "7a5a8c7e-bbcc-448f-8eb4-5a302388853d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:22:58.471223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.471193    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-dshm" (OuterVolumeSpecName: "dshm") pod "7a5a8c7e-bbcc-448f-8eb4-5a302388853d" (UID: "7a5a8c7e-bbcc-448f-8eb4-5a302388853d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:22:58.471392 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.471300    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-kube-api-access-hgdjh" (OuterVolumeSpecName: "kube-api-access-hgdjh") pod "7a5a8c7e-bbcc-448f-8eb4-5a302388853d" (UID: "7a5a8c7e-bbcc-448f-8eb4-5a302388853d"). InnerVolumeSpecName "kube-api-access-hgdjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:22:58.570599 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.570497    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgdjh\" (UniqueName: \"kubernetes.io/projected/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-kube-api-access-hgdjh\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:58.570599 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.570535    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:58.570599 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.570544    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7a5a8c7e-bbcc-448f-8eb4-5a302388853d-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:22:58.699992 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.699960    2578 generic.go:356] "Generic (PLEG): container finished" podID="7a5a8c7e-bbcc-448f-8eb4-5a302388853d" containerID="ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f" exitCode=0
Jul 30 23:22:58.700189 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.700007    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"
Jul 30 23:22:58.700189 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.700061    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn" event={"ID":"7a5a8c7e-bbcc-448f-8eb4-5a302388853d","Type":"ContainerDied","Data":"ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f"}
Jul 30 23:22:58.700189 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.700118    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn" event={"ID":"7a5a8c7e-bbcc-448f-8eb4-5a302388853d","Type":"ContainerDied","Data":"97cb2f2c605cc1cd38aa1399f033dd7b3ce71dae1924aef2cea1f822e5f07bb1"}
Jul 30 23:22:58.700189 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.700140    2578 scope.go:117] "RemoveContainer" containerID="ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f"
Jul 30 23:22:58.709358 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.709334    2578 scope.go:117] "RemoveContainer" containerID="ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f"
Jul 30 23:22:58.709651 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:22:58.709627    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f\": container with ID starting with ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f not found: ID does not exist" containerID="ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f"
Jul 30 23:22:58.709755 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.709659    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f"} err="failed to get container status \"ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f\": rpc error: code = NotFound desc = could not find container \"ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f\": container with ID starting with ebeaa84addba950dc76e39709c3de82800257adf7c12f386232656e0cfb74b4f not found: ID does not exist"
Jul 30 23:22:58.720971 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.720944    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"]
Jul 30 23:22:58.722554 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:22:58.722531    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-04f0cc/test-model-04f0cc-kserve-6677b9f5c-rnpcn"]
Jul 30 23:23:00.342241 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:00.342203    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a5a8c7e-bbcc-448f-8eb4-5a302388853d" path="/var/lib/kubelet/pods/7a5a8c7e-bbcc-448f-8eb4-5a302388853d/volumes"
Jul 30 23:23:01.072024 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.071992    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"]
Jul 30 23:23:01.072383 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072370    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a5a8c7e-bbcc-448f-8eb4-5a302388853d" containerName="main"
Jul 30 23:23:01.072431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072385    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a5a8c7e-bbcc-448f-8eb4-5a302388853d" containerName="main"
Jul 30 23:23:01.072431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072396    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4aa15b97-c2fa-49ce-9601-6763d5b3d367" containerName="payload-pre-processing"
Jul 30 23:23:01.072431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072402    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4aa15b97-c2fa-49ce-9601-6763d5b3d367" containerName="payload-pre-processing"
Jul 30 23:23:01.072431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072409    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" containerName="main"
Jul 30 23:23:01.072431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072415    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" containerName="main"
Jul 30 23:23:01.072431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072429    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69ae2e23-1846-439c-b9a7-3a4d0af360a1" containerName="payload-processing"
Jul 30 23:23:01.072613 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072434    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="69ae2e23-1846-439c-b9a7-3a4d0af360a1" containerName="payload-processing"
Jul 30 23:23:01.072613 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072495    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="69ae2e23-1846-439c-b9a7-3a4d0af360a1" containerName="payload-processing"
Jul 30 23:23:01.072613 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072502    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fc05dd8-fbf4-4b46-8e44-5c9f778ce204" containerName="main"
Jul 30 23:23:01.072613 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072510    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4aa15b97-c2fa-49ce-9601-6763d5b3d367" containerName="payload-pre-processing"
Jul 30 23:23:01.072613 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.072516    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a5a8c7e-bbcc-448f-8eb4-5a302388853d" containerName="main"
Jul 30 23:23:01.076802 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.076786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"
Jul 30 23:23:01.079282 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.079247    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jqcdc\""
Jul 30 23:23:01.082224 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.082202    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"]
Jul 30 23:23:01.194473 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.194439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp\" (UID: \"9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"
Jul 30 23:23:01.295255 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.295213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp\" (UID: \"9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"
Jul 30 23:23:01.295844 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.295824    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp\" (UID: \"9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"
Jul 30 23:23:01.388547 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.388446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"
Jul 30 23:23:01.533099 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.533072    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"]
Jul 30 23:23:01.535714 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:23:01.535684    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b74cdd2_fec1_4de1_92c8_d12a3f5fbe87.slice/crio-c200a35ee144e7f2ab57c702228f9df3fd87004a4e7364512123176d7346b764 WatchSource:0}: Error finding container c200a35ee144e7f2ab57c702228f9df3fd87004a4e7364512123176d7346b764: Status 404 returned error can't find the container with id c200a35ee144e7f2ab57c702228f9df3fd87004a4e7364512123176d7346b764
Jul 30 23:23:01.537443 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.537426    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 30 23:23:01.716423 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:01.716387    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp" event={"ID":"9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87","Type":"ContainerStarted","Data":"c200a35ee144e7f2ab57c702228f9df3fd87004a4e7364512123176d7346b764"}
Jul 30 23:23:04.737724 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:04.737690    2578 generic.go:356] "Generic (PLEG): container finished" podID="9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87" containerID="683516f2a4afbdac5753eb46e9d441724df663bc511bfff8497e602ba77f8138" exitCode=0
Jul 30 23:23:04.738134 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:04.737741    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp" event={"ID":"9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87","Type":"ContainerDied","Data":"683516f2a4afbdac5753eb46e9d441724df663bc511bfff8497e602ba77f8138"}
Jul 30 23:23:05.880173 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:05.880144    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"
Jul 30 23:23:05.939325 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:05.939292    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87-maas-api-service-ca\") pod \"9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87\" (UID: \"9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87\") "
Jul 30 23:23:05.939646 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:05.939615    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87" (UID: "9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:23:06.040868 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:06.040776    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87-maas-api-service-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:23:06.748204 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:06.748175    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"
Jul 30 23:23:06.748388 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:06.748213    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp" event={"ID":"9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87","Type":"ContainerDied","Data":"c200a35ee144e7f2ab57c702228f9df3fd87004a4e7364512123176d7346b764"}
Jul 30 23:23:06.748388 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:06.748258    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c200a35ee144e7f2ab57c702228f9df3fd87004a4e7364512123176d7346b764"
Jul 30 23:23:11.508682 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.508645    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"]
Jul 30 23:23:11.509215 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.508840    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww" podUID="99ec8eac-4cbb-4161-a86d-c5714b8ed4ae" containerName="payload-processing" containerID="cri-o://892efb9cc800a663e1219cf2df192584e1f623465c7a868bccf47a563a04a47b" gracePeriod=30
Jul 30 23:23:11.515284 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.515252    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"]
Jul 30 23:23:11.515530 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.515500    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k" podUID="40363491-9e8e-4b27-b8c9-cfec9b74b545" containerName="payload-pre-processing" containerID="cri-o://697a89f1fb4d2f151759a8127fdcc0b7c62a3a72502c0367eb71367c3b86445c" gracePeriod=30
Jul 30 23:23:11.639925 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.639892    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"]
Jul 30 23:23:11.642894 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.642864    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-04f0cc-n85kp"]
Jul 30 23:23:11.771991 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.771959    2578 generic.go:356] "Generic (PLEG): container finished" podID="40363491-9e8e-4b27-b8c9-cfec9b74b545" containerID="697a89f1fb4d2f151759a8127fdcc0b7c62a3a72502c0367eb71367c3b86445c" exitCode=0
Jul 30 23:23:11.772154 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.772077    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k" event={"ID":"40363491-9e8e-4b27-b8c9-cfec9b74b545","Type":"ContainerDied","Data":"697a89f1fb4d2f151759a8127fdcc0b7c62a3a72502c0367eb71367c3b86445c"}
Jul 30 23:23:11.773691 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.773666    2578 generic.go:356] "Generic (PLEG): container finished" podID="99ec8eac-4cbb-4161-a86d-c5714b8ed4ae" containerID="892efb9cc800a663e1219cf2df192584e1f623465c7a868bccf47a563a04a47b" exitCode=0
Jul 30 23:23:11.773787 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.773699    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww" event={"ID":"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae","Type":"ContainerDied","Data":"892efb9cc800a663e1219cf2df192584e1f623465c7a868bccf47a563a04a47b"}
Jul 30 23:23:11.773787 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.773734    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww" event={"ID":"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae","Type":"ContainerDied","Data":"d7c41470a77ee705d0bbcd848d825389c4b4c303e1c23b0b0e6c8e4135003fb3"}
Jul 30 23:23:11.773787 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.773750    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7c41470a77ee705d0bbcd848d825389c4b4c303e1c23b0b0e6c8e4135003fb3"
Jul 30 23:23:11.791271 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.791244    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:23:11.801971 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.801949    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:23:11.881502 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.881461    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40363491-9e8e-4b27-b8c9-cfec9b74b545-plugins-config-volume\") pod \"40363491-9e8e-4b27-b8c9-cfec9b74b545\" (UID: \"40363491-9e8e-4b27-b8c9-cfec9b74b545\") "
Jul 30 23:23:11.881502 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.881506    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpl9m\" (UniqueName: \"kubernetes.io/projected/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-kube-api-access-cpl9m\") pod \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\" (UID: \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\") "
Jul 30 23:23:11.881758 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.881579    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77lj8\" (UniqueName: \"kubernetes.io/projected/40363491-9e8e-4b27-b8c9-cfec9b74b545-kube-api-access-77lj8\") pod \"40363491-9e8e-4b27-b8c9-cfec9b74b545\" (UID: \"40363491-9e8e-4b27-b8c9-cfec9b74b545\") "
Jul 30 23:23:11.881758 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.881636    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-plugins-config-volume\") pod \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\" (UID: \"99ec8eac-4cbb-4161-a86d-c5714b8ed4ae\") "
Jul 30 23:23:11.881975 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.881943    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40363491-9e8e-4b27-b8c9-cfec9b74b545-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40363491-9e8e-4b27-b8c9-cfec9b74b545" (UID: "40363491-9e8e-4b27-b8c9-cfec9b74b545"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:23:11.882127 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.882036    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "99ec8eac-4cbb-4161-a86d-c5714b8ed4ae" (UID: "99ec8eac-4cbb-4161-a86d-c5714b8ed4ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:23:11.883850 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.883824    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-kube-api-access-cpl9m" (OuterVolumeSpecName: "kube-api-access-cpl9m") pod "99ec8eac-4cbb-4161-a86d-c5714b8ed4ae" (UID: "99ec8eac-4cbb-4161-a86d-c5714b8ed4ae"). InnerVolumeSpecName "kube-api-access-cpl9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:23:11.883932 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.883855    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40363491-9e8e-4b27-b8c9-cfec9b74b545-kube-api-access-77lj8" (OuterVolumeSpecName: "kube-api-access-77lj8") pod "40363491-9e8e-4b27-b8c9-cfec9b74b545" (UID: "40363491-9e8e-4b27-b8c9-cfec9b74b545"). InnerVolumeSpecName "kube-api-access-77lj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:23:11.982491 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.982453    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:23:11.982491 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.982482    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40363491-9e8e-4b27-b8c9-cfec9b74b545-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:23:11.982491 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.982491    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpl9m\" (UniqueName: \"kubernetes.io/projected/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae-kube-api-access-cpl9m\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:23:11.982491 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:11.982500    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77lj8\" (UniqueName: \"kubernetes.io/projected/40363491-9e8e-4b27-b8c9-cfec9b74b545-kube-api-access-77lj8\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:23:12.342382 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:12.342342    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87" path="/var/lib/kubelet/pods/9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87/volumes"
Jul 30 23:23:12.780163 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:12.780128    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"
Jul 30 23:23:12.780163 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:12.780145    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k" event={"ID":"40363491-9e8e-4b27-b8c9-cfec9b74b545","Type":"ContainerDied","Data":"51bf9f51531929cda55be0ac0f7aab5e29d322ab3350f6459484dddf9d3044b2"}
Jul 30 23:23:12.780163 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:12.780164    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"
Jul 30 23:23:12.780744 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:12.780240    2578 scope.go:117] "RemoveContainer" containerID="697a89f1fb4d2f151759a8127fdcc0b7c62a3a72502c0367eb71367c3b86445c"
Jul 30 23:23:12.798147 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:12.798114    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"]
Jul 30 23:23:12.801482 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:12.801455    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-04f0cc-7dd6659864-pjcww"]
Jul 30 23:23:12.811198 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:12.811168    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"]
Jul 30 23:23:12.813416 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:12.813389    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-04f0cc-65f974b644-8vw9k"]
Jul 30 23:23:14.343070 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:14.343007    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40363491-9e8e-4b27-b8c9-cfec9b74b545" path="/var/lib/kubelet/pods/40363491-9e8e-4b27-b8c9-cfec9b74b545/volumes"
Jul 30 23:23:14.343549 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:14.343518    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99ec8eac-4cbb-4161-a86d-c5714b8ed4ae" path="/var/lib/kubelet/pods/99ec8eac-4cbb-4161-a86d-c5714b8ed4ae/volumes"
Jul 30 23:23:24.450599 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.450557    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"]
Jul 30 23:23:24.451099 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.451020    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99ec8eac-4cbb-4161-a86d-c5714b8ed4ae" containerName="payload-processing"
Jul 30 23:23:24.451099 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.451038    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="99ec8eac-4cbb-4161-a86d-c5714b8ed4ae" containerName="payload-processing"
Jul 30 23:23:24.451099 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.451079    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40363491-9e8e-4b27-b8c9-cfec9b74b545" containerName="payload-pre-processing"
Jul 30 23:23:24.451099 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.451088    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="40363491-9e8e-4b27-b8c9-cfec9b74b545" containerName="payload-pre-processing"
Jul 30 23:23:24.451327 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.451105    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87" containerName="revoke-keys"
Jul 30 23:23:24.451327 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.451113    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87" containerName="revoke-keys"
Jul 30 23:23:24.451327 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.451198    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="40363491-9e8e-4b27-b8c9-cfec9b74b545" containerName="payload-pre-processing"
Jul 30 23:23:24.451327 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.451214    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b74cdd2-fec1-4de1-92c8-d12a3f5fbe87" containerName="revoke-keys"
Jul 30 23:23:24.451327 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.451224    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="99ec8eac-4cbb-4161-a86d-c5714b8ed4ae" containerName="payload-processing"
Jul 30 23:23:24.455786 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.455767    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:24.458239 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.458219    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-356bb8\""
Jul 30 23:23:24.458343 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.458258    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-356bb8-dockercfg-b5cch\""
Jul 30 23:23:24.468501 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.468477    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"]
Jul 30 23:23:24.475849 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.475814    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"]
Jul 30 23:23:24.478704 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.478669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c\" (UID: \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:24.478847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.478750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrwd9\" (UniqueName: \"kubernetes.io/projected/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-kube-api-access-wrwd9\") pod \"payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c\" (UID: \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:24.479177 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.479158    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:24.489802 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.489763    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"]
Jul 30 23:23:24.579428 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.579394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb4gq\" (UniqueName: \"kubernetes.io/projected/84f8dba9-bce2-4122-9e14-fab9781dbff4-kube-api-access-bb4gq\") pod \"payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk\" (UID: \"84f8dba9-bce2-4122-9e14-fab9781dbff4\") " pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:24.579600 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.579442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c\" (UID: \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:24.579600 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.579513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrwd9\" (UniqueName: \"kubernetes.io/projected/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-kube-api-access-wrwd9\") pod \"payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c\" (UID: \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:24.579600 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.579597    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84f8dba9-bce2-4122-9e14-fab9781dbff4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk\" (UID: \"84f8dba9-bce2-4122-9e14-fab9781dbff4\") " pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:24.580206 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.580187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c\" (UID: \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:24.587852 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.587813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrwd9\" (UniqueName: \"kubernetes.io/projected/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-kube-api-access-wrwd9\") pod \"payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c\" (UID: \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:24.680252 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.680217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84f8dba9-bce2-4122-9e14-fab9781dbff4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk\" (UID: \"84f8dba9-bce2-4122-9e14-fab9781dbff4\") " pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:24.680252 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.680270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb4gq\" (UniqueName: \"kubernetes.io/projected/84f8dba9-bce2-4122-9e14-fab9781dbff4-kube-api-access-bb4gq\") pod \"payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk\" (UID: \"84f8dba9-bce2-4122-9e14-fab9781dbff4\") " pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:24.680896 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.680875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84f8dba9-bce2-4122-9e14-fab9781dbff4-plugins-config-volume\") pod \"payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk\" (UID: \"84f8dba9-bce2-4122-9e14-fab9781dbff4\") " pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:24.688856 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.688827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb4gq\" (UniqueName: \"kubernetes.io/projected/84f8dba9-bce2-4122-9e14-fab9781dbff4-kube-api-access-bb4gq\") pod \"payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk\" (UID: \"84f8dba9-bce2-4122-9e14-fab9781dbff4\") " pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:24.765599 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.765511    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:24.790959 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.790926    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:24.946077 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.946014    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"]
Jul 30 23:23:24.950225 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:23:24.950189    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51c4a7ee_0889_4f27_817d_dc5ffc1c3a67.slice/crio-6d57c8a019dcbf05de8d72399439e59b7c259afc7b4d53ea75a8705ff36d496a WatchSource:0}: Error finding container 6d57c8a019dcbf05de8d72399439e59b7c259afc7b4d53ea75a8705ff36d496a: Status 404 returned error can't find the container with id 6d57c8a019dcbf05de8d72399439e59b7c259afc7b4d53ea75a8705ff36d496a
Jul 30 23:23:24.969955 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:24.969870    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"]
Jul 30 23:23:24.972826 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:23:24.972800    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f8dba9_bce2_4122_9e14_fab9781dbff4.slice/crio-3976afa2236e38275364ce3365b15bfcf8458c59afdfc2ddd0c585d7d36908be WatchSource:0}: Error finding container 3976afa2236e38275364ce3365b15bfcf8458c59afdfc2ddd0c585d7d36908be: Status 404 returned error can't find the container with id 3976afa2236e38275364ce3365b15bfcf8458c59afdfc2ddd0c585d7d36908be
Jul 30 23:23:25.856870 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:25.856816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c" event={"ID":"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67","Type":"ContainerStarted","Data":"9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8"}
Jul 30 23:23:25.857372 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:25.856886    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c" event={"ID":"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67","Type":"ContainerStarted","Data":"6d57c8a019dcbf05de8d72399439e59b7c259afc7b4d53ea75a8705ff36d496a"}
Jul 30 23:23:25.857372 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:25.856936    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:25.858380 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:25.858351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk" event={"ID":"84f8dba9-bce2-4122-9e14-fab9781dbff4","Type":"ContainerStarted","Data":"803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca"}
Jul 30 23:23:25.858537 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:25.858396    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk" event={"ID":"84f8dba9-bce2-4122-9e14-fab9781dbff4","Type":"ContainerStarted","Data":"3976afa2236e38275364ce3365b15bfcf8458c59afdfc2ddd0c585d7d36908be"}
Jul 30 23:23:25.858537 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:25.858426    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:25.872786 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:25.872739    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c" podStartSLOduration=1.8727233669999999 podStartE2EDuration="1.872723367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:23:25.87167897 +0000 UTC m=+2586.046642648" watchObservedRunningTime="2026-07-30 23:23:25.872723367 +0000 UTC m=+2586.047687046"
Jul 30 23:23:25.887506 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:25.887451    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk" podStartSLOduration=1.8874357609999999 podStartE2EDuration="1.887435761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:23:25.886644604 +0000 UTC m=+2586.061608296" watchObservedRunningTime="2026-07-30 23:23:25.887435761 +0000 UTC m=+2586.062399440"
Jul 30 23:23:36.866200 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:36.866167    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:23:36.866825 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:36.866360    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:23:53.549215 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.549178    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"]
Jul 30 23:23:53.552282 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.552265    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:23:53.554984 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.554960    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-1ed561-dockercfg-wdrl9\""
Jul 30 23:23:53.555151 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.555090    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-1ed561\""
Jul 30 23:23:53.563598 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.563570    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"]
Jul 30 23:23:53.594355 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.594320    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"]
Jul 30 23:23:53.597594 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.597577    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:23:53.607789 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.607761    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"]
Jul 30 23:23:53.636200 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.636167    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6825d5d3-0aa2-4857-83c1-27a66db8a71d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds\" (UID: \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:23:53.636200 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.636205    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8fm\" (UniqueName: \"kubernetes.io/projected/6825d5d3-0aa2-4857-83c1-27a66db8a71d-kube-api-access-6p8fm\") pod \"payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds\" (UID: \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:23:53.737475 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.737438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace3a109-e237-414d-a9b8-7f51978bff60-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b\" (UID: \"ace3a109-e237-414d-a9b8-7f51978bff60\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:23:53.737475 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.737478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8tkp\" (UniqueName: \"kubernetes.io/projected/ace3a109-e237-414d-a9b8-7f51978bff60-kube-api-access-h8tkp\") pod \"payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b\" (UID: \"ace3a109-e237-414d-a9b8-7f51978bff60\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:23:53.737709 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.737532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6825d5d3-0aa2-4857-83c1-27a66db8a71d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds\" (UID: \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:23:53.737709 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.737554    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8fm\" (UniqueName: \"kubernetes.io/projected/6825d5d3-0aa2-4857-83c1-27a66db8a71d-kube-api-access-6p8fm\") pod \"payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds\" (UID: \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:23:53.738281 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.738262    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6825d5d3-0aa2-4857-83c1-27a66db8a71d-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds\" (UID: \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:23:53.745512 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.745479    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8fm\" (UniqueName: \"kubernetes.io/projected/6825d5d3-0aa2-4857-83c1-27a66db8a71d-kube-api-access-6p8fm\") pod \"payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds\" (UID: \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:23:53.838643 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.838554    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8tkp\" (UniqueName: \"kubernetes.io/projected/ace3a109-e237-414d-a9b8-7f51978bff60-kube-api-access-h8tkp\") pod \"payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b\" (UID: \"ace3a109-e237-414d-a9b8-7f51978bff60\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:23:53.838643 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.838637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace3a109-e237-414d-a9b8-7f51978bff60-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b\" (UID: \"ace3a109-e237-414d-a9b8-7f51978bff60\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:23:53.839296 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.839278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace3a109-e237-414d-a9b8-7f51978bff60-plugins-config-volume\") pod \"payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b\" (UID: \"ace3a109-e237-414d-a9b8-7f51978bff60\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:23:53.846242 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.846216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8tkp\" (UniqueName: \"kubernetes.io/projected/ace3a109-e237-414d-a9b8-7f51978bff60-kube-api-access-h8tkp\") pod \"payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b\" (UID: \"ace3a109-e237-414d-a9b8-7f51978bff60\") " pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:23:53.864073 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.864024    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:23:53.909251 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:53.909216    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:23:54.013352 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:54.012978    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"]
Jul 30 23:23:54.015890 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:23:54.015855    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6825d5d3_0aa2_4857_83c1_27a66db8a71d.slice/crio-03f4b2317185194fc4b4ba11f9e385fb2f07a1e3ea02fc77caa99089a1ae440f WatchSource:0}: Error finding container 03f4b2317185194fc4b4ba11f9e385fb2f07a1e3ea02fc77caa99089a1ae440f: Status 404 returned error can't find the container with id 03f4b2317185194fc4b4ba11f9e385fb2f07a1e3ea02fc77caa99089a1ae440f
Jul 30 23:23:54.070006 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:54.069979    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"]
Jul 30 23:23:54.071761 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:23:54.071729    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podace3a109_e237_414d_a9b8_7f51978bff60.slice/crio-26aa39b761acaea277f2f12840d80224eb419c3d7563828cedd1735594c48f59 WatchSource:0}: Error finding container 26aa39b761acaea277f2f12840d80224eb419c3d7563828cedd1735594c48f59: Status 404 returned error can't find the container with id 26aa39b761acaea277f2f12840d80224eb419c3d7563828cedd1735594c48f59
Jul 30 23:23:54.996276 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:54.996233    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b" event={"ID":"ace3a109-e237-414d-a9b8-7f51978bff60","Type":"ContainerStarted","Data":"08e52fe0586411a4e0a68c951019f2e4fad3e5690a3f63a409a8c6c0382c9ee4"}
Jul 30 23:23:54.996747 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:54.996303    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:23:54.996747 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:54.996323    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b" event={"ID":"ace3a109-e237-414d-a9b8-7f51978bff60","Type":"ContainerStarted","Data":"26aa39b761acaea277f2f12840d80224eb419c3d7563828cedd1735594c48f59"}
Jul 30 23:23:54.997732 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:54.997701    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds" event={"ID":"6825d5d3-0aa2-4857-83c1-27a66db8a71d","Type":"ContainerStarted","Data":"4464da6678c93ff07e7d5e100085af4e4ae7cd8aa91e90ad591340257dc261fa"}
Jul 30 23:23:54.997855 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:54.997750    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds" event={"ID":"6825d5d3-0aa2-4857-83c1-27a66db8a71d","Type":"ContainerStarted","Data":"03f4b2317185194fc4b4ba11f9e385fb2f07a1e3ea02fc77caa99089a1ae440f"}
Jul 30 23:23:54.997855 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:54.997776    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:23:55.013308 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:55.013251    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b" podStartSLOduration=2.013236177 podStartE2EDuration="2.013236177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:23:55.011414668 +0000 UTC m=+2615.186378348" watchObservedRunningTime="2026-07-30 23:23:55.013236177 +0000 UTC m=+2615.188199856"
Jul 30 23:23:55.028785 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:23:55.028733    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds" podStartSLOduration=2.028716528 podStartE2EDuration="2.028716528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:23:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:23:55.028113056 +0000 UTC m=+2615.203076717" watchObservedRunningTime="2026-07-30 23:23:55.028716528 +0000 UTC m=+2615.203680206"
Jul 30 23:24:06.005262 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:06.005226    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:24:06.005651 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:06.005331    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:24:35.374529 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.374494    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"]
Jul 30 23:24:35.375141 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.374739    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk" podUID="84f8dba9-bce2-4122-9e14-fab9781dbff4" containerName="payload-processing" containerID="cri-o://803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca" gracePeriod=30
Jul 30 23:24:35.397499 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.390352    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"]
Jul 30 23:24:35.397499 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.390676    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c" podUID="51c4a7ee-0889-4f27-817d-dc5ffc1c3a67" containerName="payload-pre-processing" containerID="cri-o://9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8" gracePeriod=30
Jul 30 23:24:35.651605 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.651578    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:24:35.682131 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.682106    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:24:35.804501 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.804460    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrwd9\" (UniqueName: \"kubernetes.io/projected/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-kube-api-access-wrwd9\") pod \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\" (UID: \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\") "
Jul 30 23:24:35.804501 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.804505    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bb4gq\" (UniqueName: \"kubernetes.io/projected/84f8dba9-bce2-4122-9e14-fab9781dbff4-kube-api-access-bb4gq\") pod \"84f8dba9-bce2-4122-9e14-fab9781dbff4\" (UID: \"84f8dba9-bce2-4122-9e14-fab9781dbff4\") "
Jul 30 23:24:35.804772 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.804591    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-plugins-config-volume\") pod \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\" (UID: \"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67\") "
Jul 30 23:24:35.804772 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.804626    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84f8dba9-bce2-4122-9e14-fab9781dbff4-plugins-config-volume\") pod \"84f8dba9-bce2-4122-9e14-fab9781dbff4\" (UID: \"84f8dba9-bce2-4122-9e14-fab9781dbff4\") "
Jul 30 23:24:35.805033 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.805009    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "51c4a7ee-0889-4f27-817d-dc5ffc1c3a67" (UID: "51c4a7ee-0889-4f27-817d-dc5ffc1c3a67"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:24:35.805108 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.805012    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84f8dba9-bce2-4122-9e14-fab9781dbff4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "84f8dba9-bce2-4122-9e14-fab9781dbff4" (UID: "84f8dba9-bce2-4122-9e14-fab9781dbff4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:24:35.806858 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.806834    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84f8dba9-bce2-4122-9e14-fab9781dbff4-kube-api-access-bb4gq" (OuterVolumeSpecName: "kube-api-access-bb4gq") pod "84f8dba9-bce2-4122-9e14-fab9781dbff4" (UID: "84f8dba9-bce2-4122-9e14-fab9781dbff4"). InnerVolumeSpecName "kube-api-access-bb4gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:24:35.806913 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.806879    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-kube-api-access-wrwd9" (OuterVolumeSpecName: "kube-api-access-wrwd9") pod "51c4a7ee-0889-4f27-817d-dc5ffc1c3a67" (UID: "51c4a7ee-0889-4f27-817d-dc5ffc1c3a67"). InnerVolumeSpecName "kube-api-access-wrwd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:24:35.905711 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.905619    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:24:35.905711 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.905650    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/84f8dba9-bce2-4122-9e14-fab9781dbff4-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:24:35.905711 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.905664    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrwd9\" (UniqueName: \"kubernetes.io/projected/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67-kube-api-access-wrwd9\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:24:35.905711 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:35.905677    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb4gq\" (UniqueName: \"kubernetes.io/projected/84f8dba9-bce2-4122-9e14-fab9781dbff4-kube-api-access-bb4gq\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:24:36.199707 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.199667    2578 generic.go:356] "Generic (PLEG): container finished" podID="51c4a7ee-0889-4f27-817d-dc5ffc1c3a67" containerID="9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8" exitCode=0
Jul 30 23:24:36.199896 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.199714    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"
Jul 30 23:24:36.199896 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.199757    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c" event={"ID":"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67","Type":"ContainerDied","Data":"9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8"}
Jul 30 23:24:36.199896 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.199817    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c" event={"ID":"51c4a7ee-0889-4f27-817d-dc5ffc1c3a67","Type":"ContainerDied","Data":"6d57c8a019dcbf05de8d72399439e59b7c259afc7b4d53ea75a8705ff36d496a"}
Jul 30 23:24:36.199896 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.199847    2578 scope.go:117] "RemoveContainer" containerID="9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8"
Jul 30 23:24:36.201340 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.201318    2578 generic.go:356] "Generic (PLEG): container finished" podID="84f8dba9-bce2-4122-9e14-fab9781dbff4" containerID="803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca" exitCode=0
Jul 30 23:24:36.201448 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.201354    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk" event={"ID":"84f8dba9-bce2-4122-9e14-fab9781dbff4","Type":"ContainerDied","Data":"803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca"}
Jul 30 23:24:36.201448 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.201358    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"
Jul 30 23:24:36.201448 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.201393    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk" event={"ID":"84f8dba9-bce2-4122-9e14-fab9781dbff4","Type":"ContainerDied","Data":"3976afa2236e38275364ce3365b15bfcf8458c59afdfc2ddd0c585d7d36908be"}
Jul 30 23:24:36.210541 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.210521    2578 scope.go:117] "RemoveContainer" containerID="9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8"
Jul 30 23:24:36.210818 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:24:36.210786    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8\": container with ID starting with 9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8 not found: ID does not exist" containerID="9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8"
Jul 30 23:24:36.210952 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.210828    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8"} err="failed to get container status \"9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8\": rpc error: code = NotFound desc = could not find container \"9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8\": container with ID starting with 9bd122300d1fc25c11299c8f25a729cc6a130ee4e3b80f10077fc9dbb7f46fe8 not found: ID does not exist"
Jul 30 23:24:36.210952 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.210850    2578 scope.go:117] "RemoveContainer" containerID="803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca"
Jul 30 23:24:36.219457 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.219435    2578 scope.go:117] "RemoveContainer" containerID="803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca"
Jul 30 23:24:36.219730 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:24:36.219707    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca\": container with ID starting with 803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca not found: ID does not exist" containerID="803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca"
Jul 30 23:24:36.219819 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.219737    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca"} err="failed to get container status \"803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca\": rpc error: code = NotFound desc = could not find container \"803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca\": container with ID starting with 803055eecdcd54d209e51c93c049ffab1c3493969c35a70d9459a6c97c1d39ca not found: ID does not exist"
Jul 30 23:24:36.228024 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.228003    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"]
Jul 30 23:24:36.232236 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.232216    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-356bb8-6bc7dc849b-qj4mk"]
Jul 30 23:24:36.243191 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.243164    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"]
Jul 30 23:24:36.248020 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.247994    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-356bb8-6f94955596-qxv4c"]
Jul 30 23:24:36.343784 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.343724    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51c4a7ee-0889-4f27-817d-dc5ffc1c3a67" path="/var/lib/kubelet/pods/51c4a7ee-0889-4f27-817d-dc5ffc1c3a67/volumes"
Jul 30 23:24:36.344254 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:36.344235    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84f8dba9-bce2-4122-9e14-fab9781dbff4" path="/var/lib/kubelet/pods/84f8dba9-bce2-4122-9e14-fab9781dbff4/volumes"
Jul 30 23:24:42.443924 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.443882    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"]
Jul 30 23:24:42.444427 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.444387    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51c4a7ee-0889-4f27-817d-dc5ffc1c3a67" containerName="payload-pre-processing"
Jul 30 23:24:42.444427 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.444426    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="51c4a7ee-0889-4f27-817d-dc5ffc1c3a67" containerName="payload-pre-processing"
Jul 30 23:24:42.444516 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.444442    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84f8dba9-bce2-4122-9e14-fab9781dbff4" containerName="payload-processing"
Jul 30 23:24:42.444516 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.444451    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="84f8dba9-bce2-4122-9e14-fab9781dbff4" containerName="payload-processing"
Jul 30 23:24:42.444591 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.444543    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="51c4a7ee-0889-4f27-817d-dc5ffc1c3a67" containerName="payload-pre-processing"
Jul 30 23:24:42.444591 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.444555    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="84f8dba9-bce2-4122-9e14-fab9781dbff4" containerName="payload-processing"
Jul 30 23:24:42.449606 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.449590    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"
Jul 30 23:24:42.452338 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.452315    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jqcdc\""
Jul 30 23:24:42.454294 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.454274    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"]
Jul 30 23:24:42.563910 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.563876    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/604291ae-99a3-4802-934c-8c43497c2cf6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd\" (UID: \"604291ae-99a3-4802-934c-8c43497c2cf6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"
Jul 30 23:24:42.664797 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.664764    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/604291ae-99a3-4802-934c-8c43497c2cf6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd\" (UID: \"604291ae-99a3-4802-934c-8c43497c2cf6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"
Jul 30 23:24:42.665442 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.665415    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/604291ae-99a3-4802-934c-8c43497c2cf6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd\" (UID: \"604291ae-99a3-4802-934c-8c43497c2cf6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"
Jul 30 23:24:42.762068 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.761970    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"
Jul 30 23:24:42.915259 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:42.915165    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"]
Jul 30 23:24:42.917969 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:24:42.917943    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod604291ae_99a3_4802_934c_8c43497c2cf6.slice/crio-e5964b56540690884e25f548740b6e4d8d8caca851553de8d19203dd09970257 WatchSource:0}: Error finding container e5964b56540690884e25f548740b6e4d8d8caca851553de8d19203dd09970257: Status 404 returned error can't find the container with id e5964b56540690884e25f548740b6e4d8d8caca851553de8d19203dd09970257
Jul 30 23:24:43.237656 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:43.237620    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd" event={"ID":"604291ae-99a3-4802-934c-8c43497c2cf6","Type":"ContainerStarted","Data":"e5964b56540690884e25f548740b6e4d8d8caca851553de8d19203dd09970257"}
Jul 30 23:24:44.244548 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:44.244514    2578 generic.go:356] "Generic (PLEG): container finished" podID="604291ae-99a3-4802-934c-8c43497c2cf6" containerID="8f6fa53aa0d7224a8d3a77866f4fa4a7cb616a45c98629af194a91dc257e42d0" exitCode=0
Jul 30 23:24:44.244936 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:44.244602    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd" event={"ID":"604291ae-99a3-4802-934c-8c43497c2cf6","Type":"ContainerDied","Data":"8f6fa53aa0d7224a8d3a77866f4fa4a7cb616a45c98629af194a91dc257e42d0"}
Jul 30 23:24:45.381423 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:45.381394    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"
Jul 30 23:24:45.491111 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:45.491034    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/604291ae-99a3-4802-934c-8c43497c2cf6-maas-api-service-ca\") pod \"604291ae-99a3-4802-934c-8c43497c2cf6\" (UID: \"604291ae-99a3-4802-934c-8c43497c2cf6\") "
Jul 30 23:24:45.491363 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:45.491342    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/604291ae-99a3-4802-934c-8c43497c2cf6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "604291ae-99a3-4802-934c-8c43497c2cf6" (UID: "604291ae-99a3-4802-934c-8c43497c2cf6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:24:45.592635 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:45.592555    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/604291ae-99a3-4802-934c-8c43497c2cf6-maas-api-service-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:24:46.254953 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:46.254912    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd" event={"ID":"604291ae-99a3-4802-934c-8c43497c2cf6","Type":"ContainerDied","Data":"e5964b56540690884e25f548740b6e4d8d8caca851553de8d19203dd09970257"}
Jul 30 23:24:46.254953 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:46.254945    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"
Jul 30 23:24:46.255205 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:46.255010    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5964b56540690884e25f548740b6e4d8d8caca851553de8d19203dd09970257"
Jul 30 23:24:58.136483 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.136447    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"]
Jul 30 23:24:58.136980 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.136722    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b" podUID="ace3a109-e237-414d-a9b8-7f51978bff60" containerName="payload-processing" containerID="cri-o://08e52fe0586411a4e0a68c951019f2e4fad3e5690a3f63a409a8c6c0382c9ee4" gracePeriod=30
Jul 30 23:24:58.147845 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.147800    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"]
Jul 30 23:24:58.148347 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.148319    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds" podUID="6825d5d3-0aa2-4857-83c1-27a66db8a71d" containerName="payload-pre-processing" containerID="cri-o://4464da6678c93ff07e7d5e100085af4e4ae7cd8aa91e90ad591340257dc261fa" gracePeriod=30
Jul 30 23:24:58.268544 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.268302    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"]
Jul 30 23:24:58.269931 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.269899    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-1ed561-fbgwd"]
Jul 30 23:24:58.314383 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.314341    2578 generic.go:356] "Generic (PLEG): container finished" podID="ace3a109-e237-414d-a9b8-7f51978bff60" containerID="08e52fe0586411a4e0a68c951019f2e4fad3e5690a3f63a409a8c6c0382c9ee4" exitCode=0
Jul 30 23:24:58.314549 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.314405    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b" event={"ID":"ace3a109-e237-414d-a9b8-7f51978bff60","Type":"ContainerDied","Data":"08e52fe0586411a4e0a68c951019f2e4fad3e5690a3f63a409a8c6c0382c9ee4"}
Jul 30 23:24:58.316087 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.316034    2578 generic.go:356] "Generic (PLEG): container finished" podID="6825d5d3-0aa2-4857-83c1-27a66db8a71d" containerID="4464da6678c93ff07e7d5e100085af4e4ae7cd8aa91e90ad591340257dc261fa" exitCode=0
Jul 30 23:24:58.316204 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.316094    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds" event={"ID":"6825d5d3-0aa2-4857-83c1-27a66db8a71d","Type":"ContainerDied","Data":"4464da6678c93ff07e7d5e100085af4e4ae7cd8aa91e90ad591340257dc261fa"}
Jul 30 23:24:58.342482 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.342449    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="604291ae-99a3-4802-934c-8c43497c2cf6" path="/var/lib/kubelet/pods/604291ae-99a3-4802-934c-8c43497c2cf6/volumes"
Jul 30 23:24:58.462449 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.462421    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:24:58.465144 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.465123    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:24:58.596624 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.596589    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6825d5d3-0aa2-4857-83c1-27a66db8a71d-plugins-config-volume\") pod \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\" (UID: \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\") "
Jul 30 23:24:58.596624 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.596625    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8tkp\" (UniqueName: \"kubernetes.io/projected/ace3a109-e237-414d-a9b8-7f51978bff60-kube-api-access-h8tkp\") pod \"ace3a109-e237-414d-a9b8-7f51978bff60\" (UID: \"ace3a109-e237-414d-a9b8-7f51978bff60\") "
Jul 30 23:24:58.596847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.596672    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace3a109-e237-414d-a9b8-7f51978bff60-plugins-config-volume\") pod \"ace3a109-e237-414d-a9b8-7f51978bff60\" (UID: \"ace3a109-e237-414d-a9b8-7f51978bff60\") "
Jul 30 23:24:58.596847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.596719    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6p8fm\" (UniqueName: \"kubernetes.io/projected/6825d5d3-0aa2-4857-83c1-27a66db8a71d-kube-api-access-6p8fm\") pod \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\" (UID: \"6825d5d3-0aa2-4857-83c1-27a66db8a71d\") "
Jul 30 23:24:58.597037 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.597014    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6825d5d3-0aa2-4857-83c1-27a66db8a71d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6825d5d3-0aa2-4857-83c1-27a66db8a71d" (UID: "6825d5d3-0aa2-4857-83c1-27a66db8a71d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:24:58.597160 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.597126    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ace3a109-e237-414d-a9b8-7f51978bff60-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ace3a109-e237-414d-a9b8-7f51978bff60" (UID: "ace3a109-e237-414d-a9b8-7f51978bff60"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:24:58.599077 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.599021    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6825d5d3-0aa2-4857-83c1-27a66db8a71d-kube-api-access-6p8fm" (OuterVolumeSpecName: "kube-api-access-6p8fm") pod "6825d5d3-0aa2-4857-83c1-27a66db8a71d" (UID: "6825d5d3-0aa2-4857-83c1-27a66db8a71d"). InnerVolumeSpecName "kube-api-access-6p8fm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:24:58.599169 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.599043    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ace3a109-e237-414d-a9b8-7f51978bff60-kube-api-access-h8tkp" (OuterVolumeSpecName: "kube-api-access-h8tkp") pod "ace3a109-e237-414d-a9b8-7f51978bff60" (UID: "ace3a109-e237-414d-a9b8-7f51978bff60"). InnerVolumeSpecName "kube-api-access-h8tkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:24:58.697407 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.697378    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p8fm\" (UniqueName: \"kubernetes.io/projected/6825d5d3-0aa2-4857-83c1-27a66db8a71d-kube-api-access-6p8fm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:24:58.697407 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.697405    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6825d5d3-0aa2-4857-83c1-27a66db8a71d-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:24:58.697601 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.697415    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8tkp\" (UniqueName: \"kubernetes.io/projected/ace3a109-e237-414d-a9b8-7f51978bff60-kube-api-access-h8tkp\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:24:58.697601 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:58.697426    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ace3a109-e237-414d-a9b8-7f51978bff60-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:24:59.321994 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.321961    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"
Jul 30 23:24:59.322457 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.321991    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds" event={"ID":"6825d5d3-0aa2-4857-83c1-27a66db8a71d","Type":"ContainerDied","Data":"03f4b2317185194fc4b4ba11f9e385fb2f07a1e3ea02fc77caa99089a1ae440f"}
Jul 30 23:24:59.322457 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.322073    2578 scope.go:117] "RemoveContainer" containerID="4464da6678c93ff07e7d5e100085af4e4ae7cd8aa91e90ad591340257dc261fa"
Jul 30 23:24:59.323530 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.323411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b" event={"ID":"ace3a109-e237-414d-a9b8-7f51978bff60","Type":"ContainerDied","Data":"26aa39b761acaea277f2f12840d80224eb419c3d7563828cedd1735594c48f59"}
Jul 30 23:24:59.323530 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.323423    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"
Jul 30 23:24:59.331701 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.331676    2578 scope.go:117] "RemoveContainer" containerID="08e52fe0586411a4e0a68c951019f2e4fad3e5690a3f63a409a8c6c0382c9ee4"
Jul 30 23:24:59.346698 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.346671    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"]
Jul 30 23:24:59.352768 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.352746    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-1ed561-7db6f79fc4-5nc6b"]
Jul 30 23:24:59.363268 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.363238    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"]
Jul 30 23:24:59.368988 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:24:59.368966    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-1ed561-7b6b75f8dd-cr4ds"]
Jul 30 23:25:00.341460 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:00.341425    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6825d5d3-0aa2-4857-83c1-27a66db8a71d" path="/var/lib/kubelet/pods/6825d5d3-0aa2-4857-83c1-27a66db8a71d/volumes"
Jul 30 23:25:00.341841 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:00.341757    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ace3a109-e237-414d-a9b8-7f51978bff60" path="/var/lib/kubelet/pods/ace3a109-e237-414d-a9b8-7f51978bff60/volumes"
Jul 30 23:25:10.028115 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028070    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"]
Jul 30 23:25:10.028513 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028433    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ace3a109-e237-414d-a9b8-7f51978bff60" containerName="payload-processing"
Jul 30 23:25:10.028513 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028445    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ace3a109-e237-414d-a9b8-7f51978bff60" containerName="payload-processing"
Jul 30 23:25:10.028513 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028458    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6825d5d3-0aa2-4857-83c1-27a66db8a71d" containerName="payload-pre-processing"
Jul 30 23:25:10.028513 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028463    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6825d5d3-0aa2-4857-83c1-27a66db8a71d" containerName="payload-pre-processing"
Jul 30 23:25:10.028513 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028471    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="604291ae-99a3-4802-934c-8c43497c2cf6" containerName="revoke-keys"
Jul 30 23:25:10.028513 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028478    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="604291ae-99a3-4802-934c-8c43497c2cf6" containerName="revoke-keys"
Jul 30 23:25:10.028707 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028528    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6825d5d3-0aa2-4857-83c1-27a66db8a71d" containerName="payload-pre-processing"
Jul 30 23:25:10.028707 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028539    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ace3a109-e237-414d-a9b8-7f51978bff60" containerName="payload-processing"
Jul 30 23:25:10.028707 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.028548    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="604291ae-99a3-4802-934c-8c43497c2cf6" containerName="revoke-keys"
Jul 30 23:25:10.032924 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.032908    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:10.036267 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.036238    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-80952f\""
Jul 30 23:25:10.036382 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.036305    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-80952f-dockercfg-jcnz7\""
Jul 30 23:25:10.045927 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.045898    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"]
Jul 30 23:25:10.060399 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.060370    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"]
Jul 30 23:25:10.063652 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.063627    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:10.074029 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.073997    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"]
Jul 30 23:25:10.097228 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.097195    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q\" (UID: \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\") " pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:10.097367 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.097252    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b166424-9e71-4dd5-9617-f716975dfa77-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp\" (UID: \"6b166424-9e71-4dd5-9617-f716975dfa77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:10.097367 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.097297    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r9b6\" (UniqueName: \"kubernetes.io/projected/6b166424-9e71-4dd5-9617-f716975dfa77-kube-api-access-2r9b6\") pod \"payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp\" (UID: \"6b166424-9e71-4dd5-9617-f716975dfa77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:10.097367 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.097351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t68br\" (UniqueName: \"kubernetes.io/projected/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-kube-api-access-t68br\") pod \"payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q\" (UID: \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\") " pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:10.198640 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.198597    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r9b6\" (UniqueName: \"kubernetes.io/projected/6b166424-9e71-4dd5-9617-f716975dfa77-kube-api-access-2r9b6\") pod \"payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp\" (UID: \"6b166424-9e71-4dd5-9617-f716975dfa77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:10.198818 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.198681    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t68br\" (UniqueName: \"kubernetes.io/projected/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-kube-api-access-t68br\") pod \"payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q\" (UID: \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\") " pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:10.198818 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.198720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q\" (UID: \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\") " pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:10.198818 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.198784    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b166424-9e71-4dd5-9617-f716975dfa77-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp\" (UID: \"6b166424-9e71-4dd5-9617-f716975dfa77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:10.199537 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.199518    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b166424-9e71-4dd5-9617-f716975dfa77-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp\" (UID: \"6b166424-9e71-4dd5-9617-f716975dfa77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:10.199583 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.199548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q\" (UID: \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\") " pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:10.206583 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.206556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t68br\" (UniqueName: \"kubernetes.io/projected/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-kube-api-access-t68br\") pod \"payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q\" (UID: \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\") " pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:10.206788 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.206771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r9b6\" (UniqueName: \"kubernetes.io/projected/6b166424-9e71-4dd5-9617-f716975dfa77-kube-api-access-2r9b6\") pod \"payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp\" (UID: \"6b166424-9e71-4dd5-9617-f716975dfa77\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:10.342809 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.342718    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:10.375647 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.375617    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:10.497875 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.497843    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"]
Jul 30 23:25:10.502177 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:25:10.502138    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b166424_9e71_4dd5_9617_f716975dfa77.slice/crio-0f21c75dd3f771569740d7a6efb27cfc43b66328d7c0bf4778769d25517e91cb WatchSource:0}: Error finding container 0f21c75dd3f771569740d7a6efb27cfc43b66328d7c0bf4778769d25517e91cb: Status 404 returned error can't find the container with id 0f21c75dd3f771569740d7a6efb27cfc43b66328d7c0bf4778769d25517e91cb
Jul 30 23:25:10.522130 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:10.521954    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"]
Jul 30 23:25:10.532941 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:25:10.532914    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7340ccf_838e_4a29_9ed0_5c0cf6a5bcca.slice/crio-ed8e620edf8451e20171643677a169840769a091ed41ff8515982aa58f755d99 WatchSource:0}: Error finding container ed8e620edf8451e20171643677a169840769a091ed41ff8515982aa58f755d99: Status 404 returned error can't find the container with id ed8e620edf8451e20171643677a169840769a091ed41ff8515982aa58f755d99
Jul 30 23:25:11.387682 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:11.387644    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q" event={"ID":"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca","Type":"ContainerStarted","Data":"fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381"}
Jul 30 23:25:11.388176 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:11.387694    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q" event={"ID":"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca","Type":"ContainerStarted","Data":"ed8e620edf8451e20171643677a169840769a091ed41ff8515982aa58f755d99"}
Jul 30 23:25:11.388176 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:11.387725    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:11.389286 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:11.389261    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp" event={"ID":"6b166424-9e71-4dd5-9617-f716975dfa77","Type":"ContainerStarted","Data":"9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54"}
Jul 30 23:25:11.389413 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:11.389302    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp" event={"ID":"6b166424-9e71-4dd5-9617-f716975dfa77","Type":"ContainerStarted","Data":"0f21c75dd3f771569740d7a6efb27cfc43b66328d7c0bf4778769d25517e91cb"}
Jul 30 23:25:11.389413 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:11.389354    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:11.404257 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:11.404207    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q" podStartSLOduration=1.404190833 podStartE2EDuration="1.404190833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:25:11.402676155 +0000 UTC m=+2691.577639846" watchObservedRunningTime="2026-07-30 23:25:11.404190833 +0000 UTC m=+2691.579154545"
Jul 30 23:25:11.418262 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:11.418216    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp" podStartSLOduration=1.418201676 podStartE2EDuration="1.418201676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:25:11.416285029 +0000 UTC m=+2691.591248708" watchObservedRunningTime="2026-07-30 23:25:11.418201676 +0000 UTC m=+2691.593165354"
Jul 30 23:25:20.419257 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:20.419229    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:25:20.425669 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:20.425647    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:25:22.396821 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:22.396794    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:25:22.397264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:22.397019    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:25:39.084267 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.084228    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"]
Jul 30 23:25:39.087475 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.087456    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:39.090816 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.090766    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-bb1098-dockercfg-5ggrd\""
Jul 30 23:25:39.090816 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.090776    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-bb1098\""
Jul 30 23:25:39.098779 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.098745    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"]
Jul 30 23:25:39.116132 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.116100    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"]
Jul 30 23:25:39.119276 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.119259    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:25:39.129685 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.129657    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"]
Jul 30 23:25:39.259523 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.259488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14abd2ab-cd5d-4848-b181-e04c5a9c281e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42\" (UID: \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:25:39.259699 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.259600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6700b22b-3a64-42da-aa0a-86820218d15e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv\" (UID: \"6700b22b-3a64-42da-aa0a-86820218d15e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:39.259699 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.259631    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcw7p\" (UniqueName: \"kubernetes.io/projected/14abd2ab-cd5d-4848-b181-e04c5a9c281e-kube-api-access-gcw7p\") pod \"payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42\" (UID: \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:25:39.259699 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.259656    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l88rv\" (UniqueName: \"kubernetes.io/projected/6700b22b-3a64-42da-aa0a-86820218d15e-kube-api-access-l88rv\") pod \"payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv\" (UID: \"6700b22b-3a64-42da-aa0a-86820218d15e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:39.360587 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.360494    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14abd2ab-cd5d-4848-b181-e04c5a9c281e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42\" (UID: \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:25:39.360587 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.360564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6700b22b-3a64-42da-aa0a-86820218d15e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv\" (UID: \"6700b22b-3a64-42da-aa0a-86820218d15e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:39.360847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.360599    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcw7p\" (UniqueName: \"kubernetes.io/projected/14abd2ab-cd5d-4848-b181-e04c5a9c281e-kube-api-access-gcw7p\") pod \"payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42\" (UID: \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:25:39.360847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.360624    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l88rv\" (UniqueName: \"kubernetes.io/projected/6700b22b-3a64-42da-aa0a-86820218d15e-kube-api-access-l88rv\") pod \"payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv\" (UID: \"6700b22b-3a64-42da-aa0a-86820218d15e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:39.361505 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.361482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14abd2ab-cd5d-4848-b181-e04c5a9c281e-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42\" (UID: \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:25:39.361592 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.361482    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6700b22b-3a64-42da-aa0a-86820218d15e-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv\" (UID: \"6700b22b-3a64-42da-aa0a-86820218d15e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:39.369751 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.369725    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcw7p\" (UniqueName: \"kubernetes.io/projected/14abd2ab-cd5d-4848-b181-e04c5a9c281e-kube-api-access-gcw7p\") pod \"payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42\" (UID: \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\") " pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:25:39.369859 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.369836    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l88rv\" (UniqueName: \"kubernetes.io/projected/6700b22b-3a64-42da-aa0a-86820218d15e-kube-api-access-l88rv\") pod \"payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv\" (UID: \"6700b22b-3a64-42da-aa0a-86820218d15e\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:39.399624 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.399588    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:39.430949 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.430918    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:25:39.551181 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.550988    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"]
Jul 30 23:25:39.553645 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:25:39.553610    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6700b22b_3a64_42da_aa0a_86820218d15e.slice/crio-08f8deed7a8d7c05cbd24ccf76627a161effbe1246212f677cccd3df298a46af WatchSource:0}: Error finding container 08f8deed7a8d7c05cbd24ccf76627a161effbe1246212f677cccd3df298a46af: Status 404 returned error can't find the container with id 08f8deed7a8d7c05cbd24ccf76627a161effbe1246212f677cccd3df298a46af
Jul 30 23:25:39.586730 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:39.586702    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"]
Jul 30 23:25:39.593337 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:25:39.593304    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14abd2ab_cd5d_4848_b181_e04c5a9c281e.slice/crio-7a9e42a21bd8c1a2a662ba3e2349997bb0592bcbe3e16d659c60c18c6fca59f1 WatchSource:0}: Error finding container 7a9e42a21bd8c1a2a662ba3e2349997bb0592bcbe3e16d659c60c18c6fca59f1: Status 404 returned error can't find the container with id 7a9e42a21bd8c1a2a662ba3e2349997bb0592bcbe3e16d659c60c18c6fca59f1
Jul 30 23:25:40.525832 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:40.525796    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv" event={"ID":"6700b22b-3a64-42da-aa0a-86820218d15e","Type":"ContainerStarted","Data":"f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0"}
Jul 30 23:25:40.526319 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:40.525854    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv" event={"ID":"6700b22b-3a64-42da-aa0a-86820218d15e","Type":"ContainerStarted","Data":"08f8deed7a8d7c05cbd24ccf76627a161effbe1246212f677cccd3df298a46af"}
Jul 30 23:25:40.526319 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:40.525979    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:40.527390 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:40.527369    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42" event={"ID":"14abd2ab-cd5d-4848-b181-e04c5a9c281e","Type":"ContainerStarted","Data":"7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416"}
Jul 30 23:25:40.527494 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:40.527408    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:25:40.527494 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:40.527424    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42" event={"ID":"14abd2ab-cd5d-4848-b181-e04c5a9c281e","Type":"ContainerStarted","Data":"7a9e42a21bd8c1a2a662ba3e2349997bb0592bcbe3e16d659c60c18c6fca59f1"}
Jul 30 23:25:40.544494 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:40.544416    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv" podStartSLOduration=1.544399399 podStartE2EDuration="1.544399399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:25:40.542772471 +0000 UTC m=+2720.717736165" watchObservedRunningTime="2026-07-30 23:25:40.544399399 +0000 UTC m=+2720.719363078"
Jul 30 23:25:40.559211 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:40.559160    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42" podStartSLOduration=1.559141183 podStartE2EDuration="1.559141183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:25:40.557384453 +0000 UTC m=+2720.732348132" watchObservedRunningTime="2026-07-30 23:25:40.559141183 +0000 UTC m=+2720.734104884"
Jul 30 23:25:51.533763 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:51.533731    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:25:51.534362 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:25:51.533821    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:26:21.595174 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.595138    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"]
Jul 30 23:26:21.599032 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.599012    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.602489 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.602457    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-80952f\"/\"openshift-service-ca.crt\""
Jul 30 23:26:21.602648 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.602521    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-80952f\"/\"default-dockercfg-f7kkp\""
Jul 30 23:26:21.602648 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.602533    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-80952f\"/\"kube-root-ca.crt\""
Jul 30 23:26:21.602813 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.602794    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-80952f\"/\"auth-test-model-80952f-kserve-self-signed-certs\""
Jul 30 23:26:21.608950 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.608925    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"]
Jul 30 23:26:21.735122 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.735089    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-model-cache\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.735299 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.735137    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjj42\" (UniqueName: \"kubernetes.io/projected/a3fc03e6-5893-4106-a31d-afc072e4bb84-kube-api-access-sjj42\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.735299 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.735198    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-dshm\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.735299 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.735226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-home\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.735299 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.735262    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3fc03e6-5893-4106-a31d-afc072e4bb84-tls-certs\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.735299 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.735297    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-tmp-dir\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.836334 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.836301    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-home\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.836334 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.836340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3fc03e6-5893-4106-a31d-afc072e4bb84-tls-certs\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.836608 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.836364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-tmp-dir\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.836608 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.836409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-model-cache\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.836608 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.836442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjj42\" (UniqueName: \"kubernetes.io/projected/a3fc03e6-5893-4106-a31d-afc072e4bb84-kube-api-access-sjj42\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.836608 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.836493    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-dshm\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.836814 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.836772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-home\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.836814 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.836789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-model-cache\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.836913 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.836839    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-tmp-dir\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.838923 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.838895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-dshm\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.839311 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.839289    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3fc03e6-5893-4106-a31d-afc072e4bb84-tls-certs\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.845451 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.845390    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjj42\" (UniqueName: \"kubernetes.io/projected/a3fc03e6-5893-4106-a31d-afc072e4bb84-kube-api-access-sjj42\") pod \"auth-test-model-80952f-kserve-684cf4c467-ndjp8\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") " pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:21.910588 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:21.910548    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:22.049906 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:22.049879    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"]
Jul 30 23:26:22.051068 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:26:22.051018    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3fc03e6_5893_4106_a31d_afc072e4bb84.slice/crio-1d6619341bc8dace311a4717a8dba160365da2a509b40e72a212d4f74f95dbaf WatchSource:0}: Error finding container 1d6619341bc8dace311a4717a8dba160365da2a509b40e72a212d4f74f95dbaf: Status 404 returned error can't find the container with id 1d6619341bc8dace311a4717a8dba160365da2a509b40e72a212d4f74f95dbaf
Jul 30 23:26:22.730920 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:22.730888    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8" event={"ID":"a3fc03e6-5893-4106-a31d-afc072e4bb84","Type":"ContainerStarted","Data":"7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b"}
Jul 30 23:26:22.730920 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:22.730929    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8" event={"ID":"a3fc03e6-5893-4106-a31d-afc072e4bb84","Type":"ContainerStarted","Data":"1d6619341bc8dace311a4717a8dba160365da2a509b40e72a212d4f74f95dbaf"}
Jul 30 23:26:22.749744 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:22.749693    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8" podStartSLOduration=1.749674593 podStartE2EDuration="1.749674593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:26:22.748273211 +0000 UTC m=+2762.923236890" watchObservedRunningTime="2026-07-30 23:26:22.749674593 +0000 UTC m=+2762.924638271"
Jul 30 23:26:31.910932 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:31.910894    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:31.911462 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:31.911229    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:31.923365 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:31.923333    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:32.794329 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:32.794295    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:26:38.409374 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.409339    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"]
Jul 30 23:26:38.412702 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.412684    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.416455 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.416428    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"auth-test-model-bb1098-kserve-self-signed-certs\""
Jul 30 23:26:38.416455 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.416448    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"openshift-service-ca.crt\""
Jul 30 23:26:38.416654 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.416432    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"default-dockercfg-5pjtj\""
Jul 30 23:26:38.416654 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.416486    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"kube-root-ca.crt\""
Jul 30 23:26:38.422170 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.422148    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"]
Jul 30 23:26:38.477489 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.477452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn9m8\" (UniqueName: \"kubernetes.io/projected/223990df-d42e-4dd3-b579-ff0f5e289988-kube-api-access-cn9m8\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.477657 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.477497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-model-cache\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.477657 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.477526    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-home\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.477657 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.477544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-tmp-dir\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.477657 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.477612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/223990df-d42e-4dd3-b579-ff0f5e289988-tls-certs\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.477657 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.477639    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-dshm\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.578400 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.578365    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-dshm\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.578611 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.578426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn9m8\" (UniqueName: \"kubernetes.io/projected/223990df-d42e-4dd3-b579-ff0f5e289988-kube-api-access-cn9m8\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.578611 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.578478    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-model-cache\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.578611 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.578522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-home\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.578611 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.578565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-tmp-dir\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.578844 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.578641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/223990df-d42e-4dd3-b579-ff0f5e289988-tls-certs\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.578978 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.578949    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-home\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.578978 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.578973    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-tmp-dir\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.579380 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.578953    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-model-cache\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.580791 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.580769    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-dshm\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.581230 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.581210    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/223990df-d42e-4dd3-b579-ff0f5e289988-tls-certs\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.586679 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.586655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn9m8\" (UniqueName: \"kubernetes.io/projected/223990df-d42e-4dd3-b579-ff0f5e289988-kube-api-access-cn9m8\") pod \"auth-test-model-bb1098-kserve-74d7646bc5-kwxlk\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") " pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.723441 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.723410    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:38.869490 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:38.869450    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"]
Jul 30 23:26:38.872443 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:26:38.872411    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod223990df_d42e_4dd3_b579_ff0f5e289988.slice/crio-e4a7c675c240f779cf11872b615441cb63c0e8932763b2f36f68a5620f98dd21 WatchSource:0}: Error finding container e4a7c675c240f779cf11872b615441cb63c0e8932763b2f36f68a5620f98dd21: Status 404 returned error can't find the container with id e4a7c675c240f779cf11872b615441cb63c0e8932763b2f36f68a5620f98dd21
Jul 30 23:26:39.820377 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:39.820340    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk" event={"ID":"223990df-d42e-4dd3-b579-ff0f5e289988","Type":"ContainerStarted","Data":"7cfdfa75ea012ea788d0be4d3eee3f1ccdd242882019b3d0317eab4386c6ee56"}
Jul 30 23:26:39.820822 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:39.820387    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk" event={"ID":"223990df-d42e-4dd3-b579-ff0f5e289988","Type":"ContainerStarted","Data":"e4a7c675c240f779cf11872b615441cb63c0e8932763b2f36f68a5620f98dd21"}
Jul 30 23:26:39.840100 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:39.840004    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk" podStartSLOduration=1.839983637 podStartE2EDuration="1.839983637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:26:39.838426097 +0000 UTC m=+2780.013389777" watchObservedRunningTime="2026-07-30 23:26:39.839983637 +0000 UTC m=+2780.014947319"
Jul 30 23:26:48.724224 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:48.724184    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:48.724598 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:48.724237    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:48.736375 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:48.736353    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:26:48.878026 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:26:48.877998    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:27:20.604276 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:27:20.604240    2578 scope.go:117] "RemoveContainer" containerID="892efb9cc800a663e1219cf2df192584e1f623465c7a868bccf47a563a04a47b"
Jul 30 23:29:20.630674 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:29:20.630642    2578 scope.go:117] "RemoveContainer" containerID="683516f2a4afbdac5753eb46e9d441724df663bc511bfff8497e602ba77f8138"
Jul 30 23:30:02.785069 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:02.784953    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"]
Jul 30 23:30:03.700723 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:03.700690    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"]
Jul 30 23:30:10.105632 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.105594    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"]
Jul 30 23:30:10.109159 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.109141    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.111616 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.111597    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-80952f\"/\"sub-test-model-80952f-kserve-self-signed-certs\""
Jul 30 23:30:10.117368 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.117347    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"]
Jul 30 23:30:10.231141 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.231105    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-home\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.231141 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.231145    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tls-certs\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.231381 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.231183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-model-cache\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.231381 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.231202    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-dshm\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.231381 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.231246    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tmp-dir\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.231381 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.231303    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tsjj\" (UniqueName: \"kubernetes.io/projected/cc0367de-9273-49cd-ac61-0b0c2e9caca0-kube-api-access-9tsjj\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.332591 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.332558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tmp-dir\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.332591 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.332588    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tsjj\" (UniqueName: \"kubernetes.io/projected/cc0367de-9273-49cd-ac61-0b0c2e9caca0-kube-api-access-9tsjj\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.332843 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.332631    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-home\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.332843 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.332663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tls-certs\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.332843 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.332712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-model-cache\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.332843 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.332739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-dshm\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.333082 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.332997    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tmp-dir\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.333082 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.333068    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-home\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.333208 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.333101    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-model-cache\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.335155 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.335134    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-dshm\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.335398 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.335381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tls-certs\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.341368 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.341340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tsjj\" (UniqueName: \"kubernetes.io/projected/cc0367de-9273-49cd-ac61-0b0c2e9caca0-kube-api-access-9tsjj\") pod \"sub-test-model-80952f-kserve-6488d4b95f-dswjf\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") " pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.418951 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.418856    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:10.558673 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.558645    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"]
Jul 30 23:30:10.561341 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:30:10.561304    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc0367de_9273_49cd_ac61_0b0c2e9caca0.slice/crio-7b8f39c8445e508f584bb325795927ea96db1e1900947d48c5153d0e951520ac WatchSource:0}: Error finding container 7b8f39c8445e508f584bb325795927ea96db1e1900947d48c5153d0e951520ac: Status 404 returned error can't find the container with id 7b8f39c8445e508f584bb325795927ea96db1e1900947d48c5153d0e951520ac
Jul 30 23:30:10.861693 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.861658    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf" event={"ID":"cc0367de-9273-49cd-ac61-0b0c2e9caca0","Type":"ContainerStarted","Data":"c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87"}
Jul 30 23:30:10.861693 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.861702    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf" event={"ID":"cc0367de-9273-49cd-ac61-0b0c2e9caca0","Type":"ContainerStarted","Data":"7b8f39c8445e508f584bb325795927ea96db1e1900947d48c5153d0e951520ac"}
Jul 30 23:30:10.879188 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:10.879129    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf" podStartSLOduration=0.879111542 podStartE2EDuration="879.111542ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:30:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:30:10.877238284 +0000 UTC m=+2991.052201964" watchObservedRunningTime="2026-07-30 23:30:10.879111542 +0000 UTC m=+2991.054075220"
Jul 30 23:30:17.806006 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:17.805957    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8" podUID="a3fc03e6-5893-4106-a31d-afc072e4bb84" containerName="main" containerID="cri-o://7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b" gracePeriod=45
Jul 30 23:30:18.057443 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.057379    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:30:18.202800 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.202747    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3fc03e6-5893-4106-a31d-afc072e4bb84-tls-certs\") pod \"a3fc03e6-5893-4106-a31d-afc072e4bb84\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") "
Jul 30 23:30:18.202800 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.202805    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-tmp-dir\") pod \"a3fc03e6-5893-4106-a31d-afc072e4bb84\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") "
Jul 30 23:30:18.203087 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.202883    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-home\") pod \"a3fc03e6-5893-4106-a31d-afc072e4bb84\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") "
Jul 30 23:30:18.203087 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.202918    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-model-cache\") pod \"a3fc03e6-5893-4106-a31d-afc072e4bb84\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") "
Jul 30 23:30:18.203087 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.203002    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sjj42\" (UniqueName: \"kubernetes.io/projected/a3fc03e6-5893-4106-a31d-afc072e4bb84-kube-api-access-sjj42\") pod \"a3fc03e6-5893-4106-a31d-afc072e4bb84\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") "
Jul 30 23:30:18.203087 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.203035    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-dshm\") pod \"a3fc03e6-5893-4106-a31d-afc072e4bb84\" (UID: \"a3fc03e6-5893-4106-a31d-afc072e4bb84\") "
Jul 30 23:30:18.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.203198    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-home" (OuterVolumeSpecName: "home") pod "a3fc03e6-5893-4106-a31d-afc072e4bb84" (UID: "a3fc03e6-5893-4106-a31d-afc072e4bb84"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:30:18.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.203204    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a3fc03e6-5893-4106-a31d-afc072e4bb84" (UID: "a3fc03e6-5893-4106-a31d-afc072e4bb84"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:30:18.203315 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.203240    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-model-cache" (OuterVolumeSpecName: "model-cache") pod "a3fc03e6-5893-4106-a31d-afc072e4bb84" (UID: "a3fc03e6-5893-4106-a31d-afc072e4bb84"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:30:18.203423 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.203403    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:18.203423 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.203418    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:18.203489 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.203428    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:18.205367 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.205342    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-dshm" (OuterVolumeSpecName: "dshm") pod "a3fc03e6-5893-4106-a31d-afc072e4bb84" (UID: "a3fc03e6-5893-4106-a31d-afc072e4bb84"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:30:18.205484 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.205454    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3fc03e6-5893-4106-a31d-afc072e4bb84-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a3fc03e6-5893-4106-a31d-afc072e4bb84" (UID: "a3fc03e6-5893-4106-a31d-afc072e4bb84"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:30:18.205528 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.205496    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3fc03e6-5893-4106-a31d-afc072e4bb84-kube-api-access-sjj42" (OuterVolumeSpecName: "kube-api-access-sjj42") pod "a3fc03e6-5893-4106-a31d-afc072e4bb84" (UID: "a3fc03e6-5893-4106-a31d-afc072e4bb84"). InnerVolumeSpecName "kube-api-access-sjj42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:30:18.304669 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.304611    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjj42\" (UniqueName: \"kubernetes.io/projected/a3fc03e6-5893-4106-a31d-afc072e4bb84-kube-api-access-sjj42\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:18.304669 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.304666    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3fc03e6-5893-4106-a31d-afc072e4bb84-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:18.304669 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.304681    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3fc03e6-5893-4106-a31d-afc072e4bb84-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:18.721209 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.721144    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk" podUID="223990df-d42e-4dd3-b579-ff0f5e289988" containerName="main" containerID="cri-o://7cfdfa75ea012ea788d0be4d3eee3f1ccdd242882019b3d0317eab4386c6ee56" gracePeriod=45
Jul 30 23:30:18.868998 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.868945    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk" podUID="223990df-d42e-4dd3-b579-ff0f5e289988" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": dial tcp 10.134.0.72:8000: connect: connection refused"
Jul 30 23:30:18.901260 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.901230    2578 generic.go:356] "Generic (PLEG): container finished" podID="a3fc03e6-5893-4106-a31d-afc072e4bb84" containerID="7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b" exitCode=0
Jul 30 23:30:18.901447 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.901287    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"
Jul 30 23:30:18.901447 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.901362    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8" event={"ID":"a3fc03e6-5893-4106-a31d-afc072e4bb84","Type":"ContainerDied","Data":"7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b"}
Jul 30 23:30:18.901447 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.901399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8" event={"ID":"a3fc03e6-5893-4106-a31d-afc072e4bb84","Type":"ContainerDied","Data":"1d6619341bc8dace311a4717a8dba160365da2a509b40e72a212d4f74f95dbaf"}
Jul 30 23:30:18.901447 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.901420    2578 scope.go:117] "RemoveContainer" containerID="7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b"
Jul 30 23:30:18.903513 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.903484    2578 generic.go:356] "Generic (PLEG): container finished" podID="223990df-d42e-4dd3-b579-ff0f5e289988" containerID="7cfdfa75ea012ea788d0be4d3eee3f1ccdd242882019b3d0317eab4386c6ee56" exitCode=0
Jul 30 23:30:18.903651 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.903556    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk" event={"ID":"223990df-d42e-4dd3-b579-ff0f5e289988","Type":"ContainerDied","Data":"7cfdfa75ea012ea788d0be4d3eee3f1ccdd242882019b3d0317eab4386c6ee56"}
Jul 30 23:30:18.910232 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.910208    2578 scope.go:117] "RemoveContainer" containerID="7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b"
Jul 30 23:30:18.910551 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:30:18.910526    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b\": container with ID starting with 7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b not found: ID does not exist" containerID="7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b"
Jul 30 23:30:18.910632 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.910566    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b"} err="failed to get container status \"7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b\": rpc error: code = NotFound desc = could not find container \"7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b\": container with ID starting with 7854ca951edca93b9b2f42daf124121c9c4938e301fdb604ba5e1a4004d8734b not found: ID does not exist"
Jul 30 23:30:18.921886 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.921852    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"]
Jul 30 23:30:18.923660 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.923633    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/auth-test-model-80952f-kserve-684cf4c467-ndjp8"]
Jul 30 23:30:18.987211 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:18.987151    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:30:19.113418 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113388    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-dshm\") pod \"223990df-d42e-4dd3-b579-ff0f5e289988\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") "
Jul 30 23:30:19.113595 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113448    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-model-cache\") pod \"223990df-d42e-4dd3-b579-ff0f5e289988\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") "
Jul 30 23:30:19.113595 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113488    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn9m8\" (UniqueName: \"kubernetes.io/projected/223990df-d42e-4dd3-b579-ff0f5e289988-kube-api-access-cn9m8\") pod \"223990df-d42e-4dd3-b579-ff0f5e289988\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") "
Jul 30 23:30:19.113595 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113528    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/223990df-d42e-4dd3-b579-ff0f5e289988-tls-certs\") pod \"223990df-d42e-4dd3-b579-ff0f5e289988\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") "
Jul 30 23:30:19.113595 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113560    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-home\") pod \"223990df-d42e-4dd3-b579-ff0f5e289988\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") "
Jul 30 23:30:19.113595 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113592    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-tmp-dir\") pod \"223990df-d42e-4dd3-b579-ff0f5e289988\" (UID: \"223990df-d42e-4dd3-b579-ff0f5e289988\") "
Jul 30 23:30:19.113856 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113715    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-model-cache" (OuterVolumeSpecName: "model-cache") pod "223990df-d42e-4dd3-b579-ff0f5e289988" (UID: "223990df-d42e-4dd3-b579-ff0f5e289988"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:30:19.113856 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113826    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-home" (OuterVolumeSpecName: "home") pod "223990df-d42e-4dd3-b579-ff0f5e289988" (UID: "223990df-d42e-4dd3-b579-ff0f5e289988"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:30:19.113969 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113900    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "223990df-d42e-4dd3-b579-ff0f5e289988" (UID: "223990df-d42e-4dd3-b579-ff0f5e289988"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:30:19.113969 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113919    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:19.113969 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.113940    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:19.115925 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.115896    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/223990df-d42e-4dd3-b579-ff0f5e289988-kube-api-access-cn9m8" (OuterVolumeSpecName: "kube-api-access-cn9m8") pod "223990df-d42e-4dd3-b579-ff0f5e289988" (UID: "223990df-d42e-4dd3-b579-ff0f5e289988"). InnerVolumeSpecName "kube-api-access-cn9m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:30:19.116044 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.115957    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-dshm" (OuterVolumeSpecName: "dshm") pod "223990df-d42e-4dd3-b579-ff0f5e289988" (UID: "223990df-d42e-4dd3-b579-ff0f5e289988"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:30:19.116044 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.115975    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/223990df-d42e-4dd3-b579-ff0f5e289988-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "223990df-d42e-4dd3-b579-ff0f5e289988" (UID: "223990df-d42e-4dd3-b579-ff0f5e289988"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:30:19.214949 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.214916    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:19.214949 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.214946    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn9m8\" (UniqueName: \"kubernetes.io/projected/223990df-d42e-4dd3-b579-ff0f5e289988-kube-api-access-cn9m8\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:19.214949 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.214957    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/223990df-d42e-4dd3-b579-ff0f5e289988-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:19.215214 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.214966    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/223990df-d42e-4dd3-b579-ff0f5e289988-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:30:19.910838 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.910800    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk" event={"ID":"223990df-d42e-4dd3-b579-ff0f5e289988","Type":"ContainerDied","Data":"e4a7c675c240f779cf11872b615441cb63c0e8932763b2f36f68a5620f98dd21"}
Jul 30 23:30:19.910838 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.910817    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"
Jul 30 23:30:19.911388 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.910848    2578 scope.go:117] "RemoveContainer" containerID="7cfdfa75ea012ea788d0be4d3eee3f1ccdd242882019b3d0317eab4386c6ee56"
Jul 30 23:30:19.932365 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.932346    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"]
Jul 30 23:30:19.937924 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:19.937898    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/auth-test-model-bb1098-kserve-74d7646bc5-kwxlk"]
Jul 30 23:30:20.342420 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:20.342385    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="223990df-d42e-4dd3-b579-ff0f5e289988" path="/var/lib/kubelet/pods/223990df-d42e-4dd3-b579-ff0f5e289988/volumes"
Jul 30 23:30:20.342931 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:20.342913    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3fc03e6-5893-4106-a31d-afc072e4bb84" path="/var/lib/kubelet/pods/a3fc03e6-5893-4106-a31d-afc072e4bb84/volumes"
Jul 30 23:30:20.419534 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:20.419493    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:20.419534 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:20.419542    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:20.432744 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:20.432717    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:20.446766 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:20.446739    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:30:20.453827 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:20.453805    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:30:20.926315 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:20.926235    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:30:37.792931 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.792896    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"]
Jul 30 23:30:37.793430 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.793317    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3fc03e6-5893-4106-a31d-afc072e4bb84" containerName="main"
Jul 30 23:30:37.793430 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.793331    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3fc03e6-5893-4106-a31d-afc072e4bb84" containerName="main"
Jul 30 23:30:37.793430 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.793350    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="223990df-d42e-4dd3-b579-ff0f5e289988" containerName="main"
Jul 30 23:30:37.793430 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.793355    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="223990df-d42e-4dd3-b579-ff0f5e289988" containerName="main"
Jul 30 23:30:37.793430 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.793431    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3fc03e6-5893-4106-a31d-afc072e4bb84" containerName="main"
Jul 30 23:30:37.793591 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.793439    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="223990df-d42e-4dd3-b579-ff0f5e289988" containerName="main"
Jul 30 23:30:37.796970 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.796948    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.800739 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.800714    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"openshift-service-ca.crt\""
Jul 30 23:30:37.800739 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.800731    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"kube-root-ca.crt\""
Jul 30 23:30:37.800942 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.800714    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"default-dockercfg-5pjtj\""
Jul 30 23:30:37.801014 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.800995    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"sub-test-model-bb1098-kserve-self-signed-certs\""
Jul 30 23:30:37.806656 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.806632    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"]
Jul 30 23:30:37.875867 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.875833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-model-cache\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.876036 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.875874    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-home\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.876036 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.875920    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-tmp-dir\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.876036 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.875940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-dshm\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.876036 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.875955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49737631-53b4-4994-acd2-05b994c3d496-tls-certs\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.876036 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.875976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgfjx\" (UniqueName: \"kubernetes.io/projected/49737631-53b4-4994-acd2-05b994c3d496-kube-api-access-zgfjx\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.976911 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.976868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-model-cache\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.977150 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.976925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-home\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.977150 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.976972    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-tmp-dir\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.977150 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.976996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-dshm\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.977150 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.977119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49737631-53b4-4994-acd2-05b994c3d496-tls-certs\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.977388 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.977158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgfjx\" (UniqueName: \"kubernetes.io/projected/49737631-53b4-4994-acd2-05b994c3d496-kube-api-access-zgfjx\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.977388 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.977345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-model-cache\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.977481 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.977387    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-tmp-dir\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.977481 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.977406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-home\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.979465 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.979445    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-dshm\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.979736 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.979722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49737631-53b4-4994-acd2-05b994c3d496-tls-certs\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:37.985400 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:37.985378    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgfjx\" (UniqueName: \"kubernetes.io/projected/49737631-53b4-4994-acd2-05b994c3d496-kube-api-access-zgfjx\") pod \"sub-test-model-bb1098-kserve-64fb965d87-vv2zf\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") " pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:38.109022 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:38.108931    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:38.247475 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:38.247427    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"]
Jul 30 23:30:38.250661 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:30:38.250625    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49737631_53b4_4994_acd2_05b994c3d496.slice/crio-11a3426144c24cbd65bd54e95adeea9d0e51c6ee16147600fd39fe8c6cd490ed WatchSource:0}: Error finding container 11a3426144c24cbd65bd54e95adeea9d0e51c6ee16147600fd39fe8c6cd490ed: Status 404 returned error can't find the container with id 11a3426144c24cbd65bd54e95adeea9d0e51c6ee16147600fd39fe8c6cd490ed
Jul 30 23:30:39.010081 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:39.010032    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf" event={"ID":"49737631-53b4-4994-acd2-05b994c3d496","Type":"ContainerStarted","Data":"898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec"}
Jul 30 23:30:39.010081 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:39.010085    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf" event={"ID":"49737631-53b4-4994-acd2-05b994c3d496","Type":"ContainerStarted","Data":"11a3426144c24cbd65bd54e95adeea9d0e51c6ee16147600fd39fe8c6cd490ed"}
Jul 30 23:30:39.028627 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:39.028580    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf" podStartSLOduration=2.02856557 podStartE2EDuration="2.02856557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:30:39.026995542 +0000 UTC m=+3019.201959222" watchObservedRunningTime="2026-07-30 23:30:39.02856557 +0000 UTC m=+3019.203529248"
Jul 30 23:30:48.109538 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:48.109503    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:48.109927 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:48.109556    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:48.121920 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:48.121899    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:30:49.069443 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:30:49.069416    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:31:20.664499 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:31:20.664468    2578 scope.go:117] "RemoveContainer" containerID="8f6fa53aa0d7224a8d3a77866f4fa4a7cb616a45c98629af194a91dc257e42d0"
Jul 30 23:31:54.777752 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:31:54.777718    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"]
Jul 30 23:31:55.714479 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:31:55.714443    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"]
Jul 30 23:32:09.797135 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:09.797073    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf" podUID="cc0367de-9273-49cd-ac61-0b0c2e9caca0" containerName="main" containerID="cri-o://c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87" gracePeriod=45
Jul 30 23:32:10.042857 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.042832    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:32:10.109013 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.108928    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-model-cache\") pod \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") "
Jul 30 23:32:10.109013 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.108971    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-dshm\") pod \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") "
Jul 30 23:32:10.109251 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.109018    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tls-certs\") pod \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") "
Jul 30 23:32:10.109251 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.109085    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-home\") pod \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") "
Jul 30 23:32:10.109251 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.109110    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tsjj\" (UniqueName: \"kubernetes.io/projected/cc0367de-9273-49cd-ac61-0b0c2e9caca0-kube-api-access-9tsjj\") pod \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") "
Jul 30 23:32:10.109251 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.109129    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tmp-dir\") pod \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\" (UID: \"cc0367de-9273-49cd-ac61-0b0c2e9caca0\") "
Jul 30 23:32:10.109463 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.109270    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-model-cache" (OuterVolumeSpecName: "model-cache") pod "cc0367de-9273-49cd-ac61-0b0c2e9caca0" (UID: "cc0367de-9273-49cd-ac61-0b0c2e9caca0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:32:10.109463 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.109391    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:10.109571 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.109443    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-home" (OuterVolumeSpecName: "home") pod "cc0367de-9273-49cd-ac61-0b0c2e9caca0" (UID: "cc0367de-9273-49cd-ac61-0b0c2e9caca0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:32:10.109571 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.109510    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cc0367de-9273-49cd-ac61-0b0c2e9caca0" (UID: "cc0367de-9273-49cd-ac61-0b0c2e9caca0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:32:10.111231 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.111202    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-dshm" (OuterVolumeSpecName: "dshm") pod "cc0367de-9273-49cd-ac61-0b0c2e9caca0" (UID: "cc0367de-9273-49cd-ac61-0b0c2e9caca0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:32:10.111374 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.111356    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0367de-9273-49cd-ac61-0b0c2e9caca0-kube-api-access-9tsjj" (OuterVolumeSpecName: "kube-api-access-9tsjj") pod "cc0367de-9273-49cd-ac61-0b0c2e9caca0" (UID: "cc0367de-9273-49cd-ac61-0b0c2e9caca0"). InnerVolumeSpecName "kube-api-access-9tsjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:32:10.111425 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.111376    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cc0367de-9273-49cd-ac61-0b0c2e9caca0" (UID: "cc0367de-9273-49cd-ac61-0b0c2e9caca0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:32:10.210637 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.210603    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:10.210637 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.210628    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tsjj\" (UniqueName: \"kubernetes.io/projected/cc0367de-9273-49cd-ac61-0b0c2e9caca0-kube-api-access-9tsjj\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:10.210637 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.210640    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:10.210637 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.210648    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cc0367de-9273-49cd-ac61-0b0c2e9caca0-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:10.210942 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.210655    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cc0367de-9273-49cd-ac61-0b0c2e9caca0-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:10.439781 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.439749    2578 generic.go:356] "Generic (PLEG): container finished" podID="cc0367de-9273-49cd-ac61-0b0c2e9caca0" containerID="c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87" exitCode=0
Jul 30 23:32:10.439965 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.439800    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"
Jul 30 23:32:10.439965 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.439837    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf" event={"ID":"cc0367de-9273-49cd-ac61-0b0c2e9caca0","Type":"ContainerDied","Data":"c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87"}
Jul 30 23:32:10.439965 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.439880    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf" event={"ID":"cc0367de-9273-49cd-ac61-0b0c2e9caca0","Type":"ContainerDied","Data":"7b8f39c8445e508f584bb325795927ea96db1e1900947d48c5153d0e951520ac"}
Jul 30 23:32:10.439965 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.439897    2578 scope.go:117] "RemoveContainer" containerID="c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87"
Jul 30 23:32:10.448767 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.448749    2578 scope.go:117] "RemoveContainer" containerID="c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87"
Jul 30 23:32:10.449037 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:32:10.449017    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87\": container with ID starting with c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87 not found: ID does not exist" containerID="c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87"
Jul 30 23:32:10.449133 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.449069    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87"} err="failed to get container status \"c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87\": rpc error: code = NotFound desc = could not find container \"c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87\": container with ID starting with c19adce94edd13468d3b398030f457bf5271ff9859e3ad5853ded635d2c41d87 not found: ID does not exist"
Jul 30 23:32:10.455778 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.455745    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"]
Jul 30 23:32:10.457803 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.457779    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-80952f/sub-test-model-80952f-kserve-6488d4b95f-dswjf"]
Jul 30 23:32:10.734503 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.734403    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf" podUID="49737631-53b4-4994-acd2-05b994c3d496" containerName="main" containerID="cri-o://898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec" gracePeriod=45
Jul 30 23:32:10.998939 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:10.998875    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:32:11.118724 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.118690    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-model-cache\") pod \"49737631-53b4-4994-acd2-05b994c3d496\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") "
Jul 30 23:32:11.118724 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.118733    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-dshm\") pod \"49737631-53b4-4994-acd2-05b994c3d496\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") "
Jul 30 23:32:11.118990 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.118753    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49737631-53b4-4994-acd2-05b994c3d496-tls-certs\") pod \"49737631-53b4-4994-acd2-05b994c3d496\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") "
Jul 30 23:32:11.118990 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.118803    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-tmp-dir\") pod \"49737631-53b4-4994-acd2-05b994c3d496\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") "
Jul 30 23:32:11.118990 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.118860    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgfjx\" (UniqueName: \"kubernetes.io/projected/49737631-53b4-4994-acd2-05b994c3d496-kube-api-access-zgfjx\") pod \"49737631-53b4-4994-acd2-05b994c3d496\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") "
Jul 30 23:32:11.118990 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.118891    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-home\") pod \"49737631-53b4-4994-acd2-05b994c3d496\" (UID: \"49737631-53b4-4994-acd2-05b994c3d496\") "
Jul 30 23:32:11.119223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.119033    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-model-cache" (OuterVolumeSpecName: "model-cache") pod "49737631-53b4-4994-acd2-05b994c3d496" (UID: "49737631-53b4-4994-acd2-05b994c3d496"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:32:11.119223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.119172    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:11.119223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.119189    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-home" (OuterVolumeSpecName: "home") pod "49737631-53b4-4994-acd2-05b994c3d496" (UID: "49737631-53b4-4994-acd2-05b994c3d496"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:32:11.119333 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.119282    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "49737631-53b4-4994-acd2-05b994c3d496" (UID: "49737631-53b4-4994-acd2-05b994c3d496"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:32:11.121084 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.121027    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-dshm" (OuterVolumeSpecName: "dshm") pod "49737631-53b4-4994-acd2-05b994c3d496" (UID: "49737631-53b4-4994-acd2-05b994c3d496"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:32:11.121084 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.121027    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49737631-53b4-4994-acd2-05b994c3d496-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "49737631-53b4-4994-acd2-05b994c3d496" (UID: "49737631-53b4-4994-acd2-05b994c3d496"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:32:11.121231 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.121108    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49737631-53b4-4994-acd2-05b994c3d496-kube-api-access-zgfjx" (OuterVolumeSpecName: "kube-api-access-zgfjx") pod "49737631-53b4-4994-acd2-05b994c3d496" (UID: "49737631-53b4-4994-acd2-05b994c3d496"). InnerVolumeSpecName "kube-api-access-zgfjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:32:11.220246 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.220203    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgfjx\" (UniqueName: \"kubernetes.io/projected/49737631-53b4-4994-acd2-05b994c3d496-kube-api-access-zgfjx\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:11.220246 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.220239    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:11.220246 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.220252    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:11.220501 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.220263    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49737631-53b4-4994-acd2-05b994c3d496-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:11.220501 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.220276    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49737631-53b4-4994-acd2-05b994c3d496-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:32:11.446634 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.446598    2578 generic.go:356] "Generic (PLEG): container finished" podID="49737631-53b4-4994-acd2-05b994c3d496" containerID="898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec" exitCode=0
Jul 30 23:32:11.446835 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.446642    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"
Jul 30 23:32:11.446835 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.446685    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf" event={"ID":"49737631-53b4-4994-acd2-05b994c3d496","Type":"ContainerDied","Data":"898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec"}
Jul 30 23:32:11.446835 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.446731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf" event={"ID":"49737631-53b4-4994-acd2-05b994c3d496","Type":"ContainerDied","Data":"11a3426144c24cbd65bd54e95adeea9d0e51c6ee16147600fd39fe8c6cd490ed"}
Jul 30 23:32:11.446835 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.446756    2578 scope.go:117] "RemoveContainer" containerID="898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec"
Jul 30 23:32:11.457371 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.457354    2578 scope.go:117] "RemoveContainer" containerID="898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec"
Jul 30 23:32:11.457680 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:32:11.457653    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec\": container with ID starting with 898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec not found: ID does not exist" containerID="898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec"
Jul 30 23:32:11.457739 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.457693    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec"} err="failed to get container status \"898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec\": rpc error: code = NotFound desc = could not find container \"898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec\": container with ID starting with 898aa6c98d69552887fc305c137be9baa6d8d7df3708b7e78e97cf2c4eb77dec not found: ID does not exist"
Jul 30 23:32:11.468102 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.468066    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"]
Jul 30 23:32:11.471787 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:11.471760    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/sub-test-model-bb1098-kserve-64fb965d87-vv2zf"]
Jul 30 23:32:12.349669 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:12.349629    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49737631-53b4-4994-acd2-05b994c3d496" path="/var/lib/kubelet/pods/49737631-53b4-4994-acd2-05b994c3d496/volumes"
Jul 30 23:32:12.350701 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:12.350674    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc0367de-9273-49cd-ac61-0b0c2e9caca0" path="/var/lib/kubelet/pods/cc0367de-9273-49cd-ac61-0b0c2e9caca0/volumes"
Jul 30 23:32:50.802375 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.802283    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"]
Jul 30 23:32:50.802939 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.802803    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49737631-53b4-4994-acd2-05b994c3d496" containerName="main"
Jul 30 23:32:50.802939 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.802823    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="49737631-53b4-4994-acd2-05b994c3d496" containerName="main"
Jul 30 23:32:50.802939 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.802852    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0367de-9273-49cd-ac61-0b0c2e9caca0" containerName="main"
Jul 30 23:32:50.802939 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.802860    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0367de-9273-49cd-ac61-0b0c2e9caca0" containerName="main"
Jul 30 23:32:50.802939 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.802935    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0367de-9273-49cd-ac61-0b0c2e9caca0" containerName="main"
Jul 30 23:32:50.803238 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.802947    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="49737631-53b4-4994-acd2-05b994c3d496" containerName="main"
Jul 30 23:32:50.805896 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.805876    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.809608 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.809582    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"openshift-service-ca.crt\""
Jul 30 23:32:50.809758 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.809618    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"default-dockercfg-5pjtj\""
Jul 30 23:32:50.809758 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.809627    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"rate-test-model-bb1098-kserve-self-signed-certs\""
Jul 30 23:32:50.809880 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.809759    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bb1098\"/\"kube-root-ca.crt\""
Jul 30 23:32:50.815904 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.815883    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"]
Jul 30 23:32:50.865658 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.865627    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thsr4\" (UniqueName: \"kubernetes.io/projected/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-kube-api-access-thsr4\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.865827 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.865722    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-home\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.865827 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.865744    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tls-certs\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.865827 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.865783    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-dshm\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.865827 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.865815    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-model-cache\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.866000 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.865839    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tmp-dir\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.966214 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.966180    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thsr4\" (UniqueName: \"kubernetes.io/projected/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-kube-api-access-thsr4\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.966408 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.966241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-home\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.966408 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.966261    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tls-certs\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.966408 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.966283    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-dshm\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.966408 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.966305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-model-cache\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.966408 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.966319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tmp-dir\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.966689 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.966667    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tmp-dir\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.966789 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.966763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-model-cache\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.966834 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.966785    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-home\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.968753 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.968730    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-dshm\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.968966 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.968948    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tls-certs\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:50.974691 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:50.974667    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thsr4\" (UniqueName: \"kubernetes.io/projected/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-kube-api-access-thsr4\") pod \"rate-test-model-bb1098-kserve-7474bf986c-nw4sp\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") " pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:51.115991 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:51.115903    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:32:51.251836 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:51.251808    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"]
Jul 30 23:32:51.254416 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:32:51.254382    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode38621c1_79d6_4dcb_abbf_4d1dd25c8b5f.slice/crio-48015ad0700b6cc64e2886a58e20a09315f7875d5c565b707a155a5290b6cabf WatchSource:0}: Error finding container 48015ad0700b6cc64e2886a58e20a09315f7875d5c565b707a155a5290b6cabf: Status 404 returned error can't find the container with id 48015ad0700b6cc64e2886a58e20a09315f7875d5c565b707a155a5290b6cabf
Jul 30 23:32:51.638193 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:51.638156    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp" event={"ID":"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f","Type":"ContainerStarted","Data":"824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d"}
Jul 30 23:32:51.638193 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:51.638196    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp" event={"ID":"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f","Type":"ContainerStarted","Data":"48015ad0700b6cc64e2886a58e20a09315f7875d5c565b707a155a5290b6cabf"}
Jul 30 23:32:51.656645 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:32:51.656592    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp" podStartSLOduration=1.656572261 podStartE2EDuration="1.656572261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:32:51.653799283 +0000 UTC m=+3151.828762963" watchObservedRunningTime="2026-07-30 23:32:51.656572261 +0000 UTC m=+3151.831535939"
Jul 30 23:33:01.116677 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:33:01.116637    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:33:01.117123 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:33:01.116692    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:33:01.129385 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:33:01.129354    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:33:01.702003 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:33:01.701973    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:33:59.526813 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:33:59.526776    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"]
Jul 30 23:34:14.548612 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.548564    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp" podUID="e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" containerName="main" containerID="cri-o://824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d" gracePeriod=45
Jul 30 23:34:14.799664 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.799593    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:34:14.897585 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.897549    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tmp-dir\") pod \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") "
Jul 30 23:34:14.897585 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.897585    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-model-cache\") pod \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") "
Jul 30 23:34:14.897860 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.897607    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-home\") pod \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") "
Jul 30 23:34:14.897860 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.897660    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-dshm\") pod \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") "
Jul 30 23:34:14.897860 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.897701    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thsr4\" (UniqueName: \"kubernetes.io/projected/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-kube-api-access-thsr4\") pod \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") "
Jul 30 23:34:14.897860 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.897717    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tls-certs\") pod \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\" (UID: \"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f\") "
Jul 30 23:34:14.898040 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.897891    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" (UID: "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:34:14.898040 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.897913    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-model-cache" (OuterVolumeSpecName: "model-cache") pod "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" (UID: "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:34:14.898040 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.897950    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-home" (OuterVolumeSpecName: "home") pod "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" (UID: "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:34:14.899974 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.899946    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-dshm" (OuterVolumeSpecName: "dshm") pod "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" (UID: "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:34:14.900094 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.900002    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-kube-api-access-thsr4" (OuterVolumeSpecName: "kube-api-access-thsr4") pod "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" (UID: "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f"). InnerVolumeSpecName "kube-api-access-thsr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:34:14.900094 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.900010    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" (UID: "e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:34:14.999284 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.999235    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:34:14.999284 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.999277    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:34:14.999284 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.999286    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:34:14.999284 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.999294    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:34:14.999284 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.999303    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thsr4\" (UniqueName: \"kubernetes.io/projected/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-kube-api-access-thsr4\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:34:14.999595 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:14.999311    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:34:15.042063 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:15.042027    2578 generic.go:356] "Generic (PLEG): container finished" podID="e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" containerID="824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d" exitCode=0
Jul 30 23:34:15.042242 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:15.042092    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"
Jul 30 23:34:15.042242 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:15.042095    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp" event={"ID":"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f","Type":"ContainerDied","Data":"824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d"}
Jul 30 23:34:15.042242 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:15.042193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp" event={"ID":"e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f","Type":"ContainerDied","Data":"48015ad0700b6cc64e2886a58e20a09315f7875d5c565b707a155a5290b6cabf"}
Jul 30 23:34:15.042242 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:15.042209    2578 scope.go:117] "RemoveContainer" containerID="824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d"
Jul 30 23:34:15.051769 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:15.051747    2578 scope.go:117] "RemoveContainer" containerID="824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d"
Jul 30 23:34:15.052044 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:34:15.052027    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d\": container with ID starting with 824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d not found: ID does not exist" containerID="824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d"
Jul 30 23:34:15.052142 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:15.052075    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d"} err="failed to get container status \"824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d\": rpc error: code = NotFound desc = could not find container \"824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d\": container with ID starting with 824e4009b558a655fd9cb19deb90c8c98a5cbd6a195ed9241039e59c1cd2cf7d not found: ID does not exist"
Jul 30 23:34:15.062552 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:15.062527    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"]
Jul 30 23:34:15.068286 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:15.068263    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bb1098/rate-test-model-bb1098-kserve-7474bf986c-nw4sp"]
Jul 30 23:34:16.341945 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:16.341910    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" path="/var/lib/kubelet/pods/e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f/volumes"
Jul 30 23:34:24.242260 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.242226    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"]
Jul 30 23:34:24.242679 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.242632    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" containerName="main"
Jul 30 23:34:24.242679 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.242650    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" containerName="main"
Jul 30 23:34:24.242762 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.242706    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e38621c1-79d6-4dcb-abbf-4d1dd25c8b5f" containerName="main"
Jul 30 23:34:24.246958 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.246936    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:34:24.249668 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.249647    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-287423-dockercfg-kskpd\""
Jul 30 23:34:24.249786 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.249648    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-287423\""
Jul 30 23:34:24.261666 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.261640    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"]
Jul 30 23:34:24.274637 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.274603    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"]
Jul 30 23:34:24.277968 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.277942    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:24.289095 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.289020    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"]
Jul 30 23:34:24.289265 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.289088    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh\" (UID: \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:24.289345 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.289284    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-678px\" (UniqueName: \"kubernetes.io/projected/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-kube-api-access-678px\") pod \"payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh\" (UID: \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:24.390196 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.390165    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh\" (UID: \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:24.390382 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.390217    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c51e275-4b42-41db-aa3c-22f09ff9605a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb\" (UID: \"7c51e275-4b42-41db-aa3c-22f09ff9605a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:34:24.390382 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.390274    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-678px\" (UniqueName: \"kubernetes.io/projected/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-kube-api-access-678px\") pod \"payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh\" (UID: \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:24.390382 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.390373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-679g4\" (UniqueName: \"kubernetes.io/projected/7c51e275-4b42-41db-aa3c-22f09ff9605a-kube-api-access-679g4\") pod \"payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb\" (UID: \"7c51e275-4b42-41db-aa3c-22f09ff9605a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:34:24.390920 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.390898    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh\" (UID: \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:24.399321 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.399294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-678px\" (UniqueName: \"kubernetes.io/projected/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-kube-api-access-678px\") pod \"payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh\" (UID: \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:24.491720 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.491683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679g4\" (UniqueName: \"kubernetes.io/projected/7c51e275-4b42-41db-aa3c-22f09ff9605a-kube-api-access-679g4\") pod \"payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb\" (UID: \"7c51e275-4b42-41db-aa3c-22f09ff9605a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:34:24.491903 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.491757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c51e275-4b42-41db-aa3c-22f09ff9605a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb\" (UID: \"7c51e275-4b42-41db-aa3c-22f09ff9605a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:34:24.492473 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.492427    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c51e275-4b42-41db-aa3c-22f09ff9605a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb\" (UID: \"7c51e275-4b42-41db-aa3c-22f09ff9605a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:34:24.501360 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.501331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-679g4\" (UniqueName: \"kubernetes.io/projected/7c51e275-4b42-41db-aa3c-22f09ff9605a-kube-api-access-679g4\") pod \"payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb\" (UID: \"7c51e275-4b42-41db-aa3c-22f09ff9605a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:34:24.556875 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.556843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:34:24.590140 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.590104    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:24.708039 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.707809    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"]
Jul 30 23:34:24.710602 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:34:24.710576    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c51e275_4b42_41db_aa3c_22f09ff9605a.slice/crio-f38fd2ccc750e411f68a11b69a8ed09fa099ed7afccfebc64b6e4803e89404ec WatchSource:0}: Error finding container f38fd2ccc750e411f68a11b69a8ed09fa099ed7afccfebc64b6e4803e89404ec: Status 404 returned error can't find the container with id f38fd2ccc750e411f68a11b69a8ed09fa099ed7afccfebc64b6e4803e89404ec
Jul 30 23:34:24.956042 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:24.956005    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"]
Jul 30 23:34:24.958270 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:34:24.958234    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecff7c9a_27d0_416a_88a6_fe64dfaef58f.slice/crio-183f7a1f33f2a67fc687458c94997f605e5540df7c0e92242599e9b37e0a4e80 WatchSource:0}: Error finding container 183f7a1f33f2a67fc687458c94997f605e5540df7c0e92242599e9b37e0a4e80: Status 404 returned error can't find the container with id 183f7a1f33f2a67fc687458c94997f605e5540df7c0e92242599e9b37e0a4e80
Jul 30 23:34:25.093379 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:25.093330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh" event={"ID":"ecff7c9a-27d0-416a-88a6-fe64dfaef58f","Type":"ContainerStarted","Data":"50fe2ea9957441f5da7f331363701ed84647ddb1335de76477653c132f444550"}
Jul 30 23:34:25.093568 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:25.093399    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:25.093568 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:25.093426    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh" event={"ID":"ecff7c9a-27d0-416a-88a6-fe64dfaef58f","Type":"ContainerStarted","Data":"183f7a1f33f2a67fc687458c94997f605e5540df7c0e92242599e9b37e0a4e80"}
Jul 30 23:34:25.094815 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:25.094784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb" event={"ID":"7c51e275-4b42-41db-aa3c-22f09ff9605a","Type":"ContainerStarted","Data":"bb8a46863baea5d3ae004f3db49813c0b7666b7d4205f3a149cc7a9c1ce79240"}
Jul 30 23:34:25.094939 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:25.094833    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb" event={"ID":"7c51e275-4b42-41db-aa3c-22f09ff9605a","Type":"ContainerStarted","Data":"f38fd2ccc750e411f68a11b69a8ed09fa099ed7afccfebc64b6e4803e89404ec"}
Jul 30 23:34:25.094939 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:25.094896    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:34:25.111020 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:25.110967    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh" podStartSLOduration=1.110950752 podStartE2EDuration="1.110950752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:34:25.109410843 +0000 UTC m=+3245.284374519" watchObservedRunningTime="2026-07-30 23:34:25.110950752 +0000 UTC m=+3245.285914430"
Jul 30 23:34:25.125988 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:25.125938    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb" podStartSLOduration=1.125921225 podStartE2EDuration="1.125921225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:34:25.123919484 +0000 UTC m=+3245.298883197" watchObservedRunningTime="2026-07-30 23:34:25.125921225 +0000 UTC m=+3245.300884898"
Jul 30 23:34:36.102159 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:36.102111    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb" podUID="7c51e275-4b42-41db-aa3c-22f09ff9605a" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.76:9004: connect: connection refused"
Jul 30 23:34:36.102620 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:36.102455    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:34:44.557702 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:44.557650    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb" podUID="7c51e275-4b42-41db-aa3c-22f09ff9605a" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.76:9004: connect: connection refused"
Jul 30 23:34:46.100360 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:46.100313    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb" podUID="7c51e275-4b42-41db-aa3c-22f09ff9605a" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.76:9004: connect: connection refused"
Jul 30 23:34:56.100856 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:34:56.100825    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:35:03.726413 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.726370    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"]
Jul 30 23:35:03.729766 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.729744    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:03.732724 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.732700    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-02337c-dockercfg-kzb8f\""
Jul 30 23:35:03.732841 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.732700    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-02337c\""
Jul 30 23:35:03.740555 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.740526    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"]
Jul 30 23:35:03.751700 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.751633    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"]
Jul 30 23:35:03.755793 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.755759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:03.764074 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.763706    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"]
Jul 30 23:35:03.826890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.826850    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcmv6\" (UniqueName: \"kubernetes.io/projected/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-kube-api-access-xcmv6\") pod \"payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g\" (UID: \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:03.827135 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.826940    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bth8g\" (UniqueName: \"kubernetes.io/projected/4ab6814c-58eb-4115-9375-105fa30afde9-kube-api-access-bth8g\") pod \"payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2\" (UID: \"4ab6814c-58eb-4115-9375-105fa30afde9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:03.827135 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.826964    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ab6814c-58eb-4115-9375-105fa30afde9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2\" (UID: \"4ab6814c-58eb-4115-9375-105fa30afde9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:03.827135 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.826985    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g\" (UID: \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:03.928257 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.928213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bth8g\" (UniqueName: \"kubernetes.io/projected/4ab6814c-58eb-4115-9375-105fa30afde9-kube-api-access-bth8g\") pod \"payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2\" (UID: \"4ab6814c-58eb-4115-9375-105fa30afde9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:03.928257 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.928260    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ab6814c-58eb-4115-9375-105fa30afde9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2\" (UID: \"4ab6814c-58eb-4115-9375-105fa30afde9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:03.928497 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.928286    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g\" (UID: \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:03.928497 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.928337    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcmv6\" (UniqueName: \"kubernetes.io/projected/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-kube-api-access-xcmv6\") pod \"payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g\" (UID: \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:03.929036 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.929015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g\" (UID: \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:03.929324 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.929306    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ab6814c-58eb-4115-9375-105fa30afde9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2\" (UID: \"4ab6814c-58eb-4115-9375-105fa30afde9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:03.937261 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.937222    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcmv6\" (UniqueName: \"kubernetes.io/projected/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-kube-api-access-xcmv6\") pod \"payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g\" (UID: \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:03.937261 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:03.937226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bth8g\" (UniqueName: \"kubernetes.io/projected/4ab6814c-58eb-4115-9375-105fa30afde9-kube-api-access-bth8g\") pod \"payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2\" (UID: \"4ab6814c-58eb-4115-9375-105fa30afde9\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:04.042236 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:04.042144    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:04.068758 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:04.068719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:04.215601 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:04.215520    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"]
Jul 30 23:35:04.217753 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:35:04.217717    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab6814c_58eb_4115_9375_105fa30afde9.slice/crio-f91222276b558c0fd47f9e9cd064501372ffd4eeb748dfec584086e1180ad9a3 WatchSource:0}: Error finding container f91222276b558c0fd47f9e9cd064501372ffd4eeb748dfec584086e1180ad9a3: Status 404 returned error can't find the container with id f91222276b558c0fd47f9e9cd064501372ffd4eeb748dfec584086e1180ad9a3
Jul 30 23:35:04.243024 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:04.242994    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"]
Jul 30 23:35:04.246765 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:35:04.246730    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b9ac5f7_d615_4ce1_88d9_48c3bec0e54e.slice/crio-52534018ba3a249ee879d3e39a6382523fbabe1aface6ff83d19b34f54eafc3f WatchSource:0}: Error finding container 52534018ba3a249ee879d3e39a6382523fbabe1aface6ff83d19b34f54eafc3f: Status 404 returned error can't find the container with id 52534018ba3a249ee879d3e39a6382523fbabe1aface6ff83d19b34f54eafc3f
Jul 30 23:35:04.288703 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:04.288666    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2" event={"ID":"4ab6814c-58eb-4115-9375-105fa30afde9","Type":"ContainerStarted","Data":"f91222276b558c0fd47f9e9cd064501372ffd4eeb748dfec584086e1180ad9a3"}
Jul 30 23:35:04.289886 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:04.289860    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g" event={"ID":"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e","Type":"ContainerStarted","Data":"52534018ba3a249ee879d3e39a6382523fbabe1aface6ff83d19b34f54eafc3f"}
Jul 30 23:35:05.296539 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:05.296495    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2" event={"ID":"4ab6814c-58eb-4115-9375-105fa30afde9","Type":"ContainerStarted","Data":"53bfe861ac9ae92f6c62d34e866fe99aa5c1c429347ce1ba34b6df120ae1e9df"}
Jul 30 23:35:05.297023 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:05.296713    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:05.298093 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:05.298041    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g" event={"ID":"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e","Type":"ContainerStarted","Data":"8d660f229eb0ba3ed74f8afd8c9e1b4bf4009af6636876599718f3aac8395e5e"}
Jul 30 23:35:05.298198 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:05.298179    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:05.313208 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:05.313150    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2" podStartSLOduration=2.313130768 podStartE2EDuration="2.313130768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:35:05.311276362 +0000 UTC m=+3285.486240041" watchObservedRunningTime="2026-07-30 23:35:05.313130768 +0000 UTC m=+3285.488094447"
Jul 30 23:35:05.327994 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:05.327942    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g" podStartSLOduration=2.327924709 podStartE2EDuration="2.327924709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:35:05.327091392 +0000 UTC m=+3285.502055068" watchObservedRunningTime="2026-07-30 23:35:05.327924709 +0000 UTC m=+3285.502888388"
Jul 30 23:35:16.305253 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:16.305219    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:35:16.305849 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:16.305527    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:35:20.474471 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:20.474443    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:35:20.481334 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:20.481313    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xj7kf_7cf62e27-4b97-435a-85d0-cb7b09b2040d/ovn-acl-logging/0.log"
Jul 30 23:35:33.953531 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:33.953480    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-n2d2v_b9801adf-19b8-4d27-b493-69577a9e4c33/payload-processing/0.log"
Jul 30 23:35:34.073467 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:34.073432    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g_5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e/payload-processing/0.log"
Jul 30 23:35:37.792838 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.792801    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"]
Jul 30 23:35:37.796864 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.796843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.799405 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.799383    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-287423\"/\"default-dockercfg-fwwkg\""
Jul 30 23:35:37.800431 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.800415    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-287423\"/\"kube-root-ca.crt\""
Jul 30 23:35:37.800581 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.800562    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-287423\"/\"ipp-route-287423-kserve-self-signed-certs\""
Jul 30 23:35:37.800636 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.800582    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-287423\"/\"openshift-service-ca.crt\""
Jul 30 23:35:37.805652 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.805631    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"]
Jul 30 23:35:37.828289 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.828254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-tmp-dir\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.828482 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.828398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19837b0f-6578-4e47-adf9-f99c70e32afe-tls-certs\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.828482 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.828466    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-home\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.828621 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.828541    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdq58\" (UniqueName: \"kubernetes.io/projected/19837b0f-6578-4e47-adf9-f99c70e32afe-kube-api-access-bdq58\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.828621 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.828571    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-model-cache\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.828621 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.828600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-dshm\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.929335 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.929297    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdq58\" (UniqueName: \"kubernetes.io/projected/19837b0f-6578-4e47-adf9-f99c70e32afe-kube-api-access-bdq58\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.929335 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.929340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-model-cache\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.929615 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.929360    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-dshm\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.929615 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.929386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-tmp-dir\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.929615 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.929428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19837b0f-6578-4e47-adf9-f99c70e32afe-tls-certs\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.929615 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.929451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-home\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.929861 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.929806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-model-cache\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.929912 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.929866    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-home\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.929912 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.929881    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-tmp-dir\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.931789 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.931764    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-dshm\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.932038 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.932023    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19837b0f-6578-4e47-adf9-f99c70e32afe-tls-certs\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:37.937565 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:37.937538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdq58\" (UniqueName: \"kubernetes.io/projected/19837b0f-6578-4e47-adf9-f99c70e32afe-kube-api-access-bdq58\") pod \"ipp-route-287423-kserve-57fd54d59-jw26t\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") " pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:38.107409 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:38.107326    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:38.244160 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:38.244085    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"]
Jul 30 23:35:38.246740 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:35:38.246710    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19837b0f_6578_4e47_adf9_f99c70e32afe.slice/crio-e90308ccc769b87acf16ed67c52dfe437f0fbb28f071636ed13c79dc2702ba13 WatchSource:0}: Error finding container e90308ccc769b87acf16ed67c52dfe437f0fbb28f071636ed13c79dc2702ba13: Status 404 returned error can't find the container with id e90308ccc769b87acf16ed67c52dfe437f0fbb28f071636ed13c79dc2702ba13
Jul 30 23:35:38.460679 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:38.460646    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t" event={"ID":"19837b0f-6578-4e47-adf9-f99c70e32afe","Type":"ContainerStarted","Data":"4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee"}
Jul 30 23:35:38.460892 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:38.460689    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t" event={"ID":"19837b0f-6578-4e47-adf9-f99c70e32afe","Type":"ContainerStarted","Data":"e90308ccc769b87acf16ed67c52dfe437f0fbb28f071636ed13c79dc2702ba13"}
Jul 30 23:35:38.476620 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:38.476571    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t" podStartSLOduration=1.47655431 podStartE2EDuration="1.47655431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:35:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:35:38.47601189 +0000 UTC m=+3318.650975561" watchObservedRunningTime="2026-07-30 23:35:38.47655431 +0000 UTC m=+3318.651517989"
Jul 30 23:35:48.108528 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:48.108437    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:48.108528 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:48.108495    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:48.121015 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:48.120989    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:35:48.522974 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:35:48.522946    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:36:13.776986 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:13.776949    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh_ecff7c9a-27d0-416a-88a6-fe64dfaef58f/payload-processing/0.log"
Jul 30 23:36:13.910571 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:13.910531    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-n2d2v_b9801adf-19b8-4d27-b493-69577a9e4c33/payload-processing/0.log"
Jul 30 23:36:14.036320 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:14.036239    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g_5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e/payload-processing/0.log"
Jul 30 23:36:19.054727 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.054691    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"]
Jul 30 23:36:19.058335 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.058309    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:19.061349 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.061327    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-6a2f66-dockercfg-szgg5\""
Jul 30 23:36:19.070605 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.070585    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-6a2f66\""
Jul 30 23:36:19.082287 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.082259    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"]
Jul 30 23:36:19.109100 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.109044    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7356341-92e7-424a-aa74-d52d63f23ea7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr\" (UID: \"b7356341-92e7-424a-aa74-d52d63f23ea7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:19.109338 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.109113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9nkd\" (UniqueName: \"kubernetes.io/projected/b7356341-92e7-424a-aa74-d52d63f23ea7-kube-api-access-q9nkd\") pod \"payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr\" (UID: \"b7356341-92e7-424a-aa74-d52d63f23ea7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:19.152200 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.152169    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"]
Jul 30 23:36:19.155816 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.155795    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:19.166796 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.166770    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"]
Jul 30 23:36:19.209542 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.209493    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9nkd\" (UniqueName: \"kubernetes.io/projected/b7356341-92e7-424a-aa74-d52d63f23ea7-kube-api-access-q9nkd\") pod \"payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr\" (UID: \"b7356341-92e7-424a-aa74-d52d63f23ea7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:19.209734 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.209569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdfbn\" (UniqueName: \"kubernetes.io/projected/8819248a-b2ef-439f-9197-581fcad3f278-kube-api-access-wdfbn\") pod \"payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t\" (UID: \"8819248a-b2ef-439f-9197-581fcad3f278\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:19.209734 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.209610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8819248a-b2ef-439f-9197-581fcad3f278-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t\" (UID: \"8819248a-b2ef-439f-9197-581fcad3f278\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:19.209734 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.209646    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7356341-92e7-424a-aa74-d52d63f23ea7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr\" (UID: \"b7356341-92e7-424a-aa74-d52d63f23ea7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:19.210349 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.210331    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7356341-92e7-424a-aa74-d52d63f23ea7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr\" (UID: \"b7356341-92e7-424a-aa74-d52d63f23ea7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:19.219797 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.219773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9nkd\" (UniqueName: \"kubernetes.io/projected/b7356341-92e7-424a-aa74-d52d63f23ea7-kube-api-access-q9nkd\") pod \"payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr\" (UID: \"b7356341-92e7-424a-aa74-d52d63f23ea7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:19.310841 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.310754    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8819248a-b2ef-439f-9197-581fcad3f278-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t\" (UID: \"8819248a-b2ef-439f-9197-581fcad3f278\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:19.310996 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.310848    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdfbn\" (UniqueName: \"kubernetes.io/projected/8819248a-b2ef-439f-9197-581fcad3f278-kube-api-access-wdfbn\") pod \"payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t\" (UID: \"8819248a-b2ef-439f-9197-581fcad3f278\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:19.311520 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.311499    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8819248a-b2ef-439f-9197-581fcad3f278-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t\" (UID: \"8819248a-b2ef-439f-9197-581fcad3f278\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:19.319336 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.319318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdfbn\" (UniqueName: \"kubernetes.io/projected/8819248a-b2ef-439f-9197-581fcad3f278-kube-api-access-wdfbn\") pod \"payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t\" (UID: \"8819248a-b2ef-439f-9197-581fcad3f278\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:19.368954 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.368918    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:19.467396 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.467359    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:19.517039 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.517005    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"]
Jul 30 23:36:19.517714 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:36:19.517685    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7356341_92e7_424a_aa74_d52d63f23ea7.slice/crio-a9a0b205641a9b4c20e556b46bfcaf2d3489ad140f4fdc8b909cf1dd4b61ce9b WatchSource:0}: Error finding container a9a0b205641a9b4c20e556b46bfcaf2d3489ad140f4fdc8b909cf1dd4b61ce9b: Status 404 returned error can't find the container with id a9a0b205641a9b4c20e556b46bfcaf2d3489ad140f4fdc8b909cf1dd4b61ce9b
Jul 30 23:36:19.621306 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.621275    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"]
Jul 30 23:36:19.625207 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:36:19.625178    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8819248a_b2ef_439f_9197_581fcad3f278.slice/crio-ce7ef50835d2f98fbc64efff9853ad6b633e74bf3a08c94b4939197beb9c776a WatchSource:0}: Error finding container ce7ef50835d2f98fbc64efff9853ad6b633e74bf3a08c94b4939197beb9c776a: Status 404 returned error can't find the container with id ce7ef50835d2f98fbc64efff9853ad6b633e74bf3a08c94b4939197beb9c776a
Jul 30 23:36:19.668108 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.668043    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t" event={"ID":"8819248a-b2ef-439f-9197-581fcad3f278","Type":"ContainerStarted","Data":"ce7ef50835d2f98fbc64efff9853ad6b633e74bf3a08c94b4939197beb9c776a"}
Jul 30 23:36:19.669616 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.669590    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr" event={"ID":"b7356341-92e7-424a-aa74-d52d63f23ea7","Type":"ContainerStarted","Data":"6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564"}
Jul 30 23:36:19.669759 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.669634    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr" event={"ID":"b7356341-92e7-424a-aa74-d52d63f23ea7","Type":"ContainerStarted","Data":"a9a0b205641a9b4c20e556b46bfcaf2d3489ad140f4fdc8b909cf1dd4b61ce9b"}
Jul 30 23:36:19.669759 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.669692    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:19.686102 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:19.686032    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr" podStartSLOduration=0.686009078 podStartE2EDuration="686.009078ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:36:19.684586076 +0000 UTC m=+3359.859549755" watchObservedRunningTime="2026-07-30 23:36:19.686009078 +0000 UTC m=+3359.860972758"
Jul 30 23:36:20.676024 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:20.675984    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t" event={"ID":"8819248a-b2ef-439f-9197-581fcad3f278","Type":"ContainerStarted","Data":"1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc"}
Jul 30 23:36:20.693104 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:20.693038    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t" podStartSLOduration=1.6930208009999999 podStartE2EDuration="1.693020801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:36:20.690947609 +0000 UTC m=+3360.865911290" watchObservedRunningTime="2026-07-30 23:36:20.693020801 +0000 UTC m=+3360.867984479"
Jul 30 23:36:21.680871 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:21.680835    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:30.678575 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:30.678544    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:32.687893 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:32.687865    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:48.800612 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:48.800573    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"]
Jul 30 23:36:48.801248 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:48.800839    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr" podUID="b7356341-92e7-424a-aa74-d52d63f23ea7" containerName="payload-pre-processing" containerID="cri-o://6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564" gracePeriod=30
Jul 30 23:36:48.803083 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:48.803061    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"]
Jul 30 23:36:48.803276 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:48.803238    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t" podUID="8819248a-b2ef-439f-9197-581fcad3f278" containerName="payload-processing" containerID="cri-o://1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc" gracePeriod=30
Jul 30 23:36:49.110572 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.110546    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:49.114111 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.114087    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:49.285872 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.285833    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8819248a-b2ef-439f-9197-581fcad3f278-plugins-config-volume\") pod \"8819248a-b2ef-439f-9197-581fcad3f278\" (UID: \"8819248a-b2ef-439f-9197-581fcad3f278\") "
Jul 30 23:36:49.286088 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.285891    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7356341-92e7-424a-aa74-d52d63f23ea7-plugins-config-volume\") pod \"b7356341-92e7-424a-aa74-d52d63f23ea7\" (UID: \"b7356341-92e7-424a-aa74-d52d63f23ea7\") "
Jul 30 23:36:49.286088 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.285935    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdfbn\" (UniqueName: \"kubernetes.io/projected/8819248a-b2ef-439f-9197-581fcad3f278-kube-api-access-wdfbn\") pod \"8819248a-b2ef-439f-9197-581fcad3f278\" (UID: \"8819248a-b2ef-439f-9197-581fcad3f278\") "
Jul 30 23:36:49.286088 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.285992    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9nkd\" (UniqueName: \"kubernetes.io/projected/b7356341-92e7-424a-aa74-d52d63f23ea7-kube-api-access-q9nkd\") pod \"b7356341-92e7-424a-aa74-d52d63f23ea7\" (UID: \"b7356341-92e7-424a-aa74-d52d63f23ea7\") "
Jul 30 23:36:49.286348 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.286325    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8819248a-b2ef-439f-9197-581fcad3f278-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8819248a-b2ef-439f-9197-581fcad3f278" (UID: "8819248a-b2ef-439f-9197-581fcad3f278"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:36:49.286418 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.286350    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7356341-92e7-424a-aa74-d52d63f23ea7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7356341-92e7-424a-aa74-d52d63f23ea7" (UID: "b7356341-92e7-424a-aa74-d52d63f23ea7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:36:49.288360 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.288321    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8819248a-b2ef-439f-9197-581fcad3f278-kube-api-access-wdfbn" (OuterVolumeSpecName: "kube-api-access-wdfbn") pod "8819248a-b2ef-439f-9197-581fcad3f278" (UID: "8819248a-b2ef-439f-9197-581fcad3f278"). InnerVolumeSpecName "kube-api-access-wdfbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:36:49.288477 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.288326    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7356341-92e7-424a-aa74-d52d63f23ea7-kube-api-access-q9nkd" (OuterVolumeSpecName: "kube-api-access-q9nkd") pod "b7356341-92e7-424a-aa74-d52d63f23ea7" (UID: "b7356341-92e7-424a-aa74-d52d63f23ea7"). InnerVolumeSpecName "kube-api-access-q9nkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:36:49.387907 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.387817    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8819248a-b2ef-439f-9197-581fcad3f278-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:36:49.387907 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.387856    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7356341-92e7-424a-aa74-d52d63f23ea7-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:36:49.387907 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.387875    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdfbn\" (UniqueName: \"kubernetes.io/projected/8819248a-b2ef-439f-9197-581fcad3f278-kube-api-access-wdfbn\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:36:49.387907 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.387887    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9nkd\" (UniqueName: \"kubernetes.io/projected/b7356341-92e7-424a-aa74-d52d63f23ea7-kube-api-access-q9nkd\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:36:49.827009 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.826976    2578 generic.go:356] "Generic (PLEG): container finished" podID="8819248a-b2ef-439f-9197-581fcad3f278" containerID="1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc" exitCode=0
Jul 30 23:36:49.827445 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.827026    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"
Jul 30 23:36:49.827445 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.827075    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t" event={"ID":"8819248a-b2ef-439f-9197-581fcad3f278","Type":"ContainerDied","Data":"1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc"}
Jul 30 23:36:49.827445 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.827130    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t" event={"ID":"8819248a-b2ef-439f-9197-581fcad3f278","Type":"ContainerDied","Data":"ce7ef50835d2f98fbc64efff9853ad6b633e74bf3a08c94b4939197beb9c776a"}
Jul 30 23:36:49.827445 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.827157    2578 scope.go:117] "RemoveContainer" containerID="1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc"
Jul 30 23:36:49.828419 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.828298    2578 generic.go:356] "Generic (PLEG): container finished" podID="b7356341-92e7-424a-aa74-d52d63f23ea7" containerID="6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564" exitCode=0
Jul 30 23:36:49.828419 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.828326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr" event={"ID":"b7356341-92e7-424a-aa74-d52d63f23ea7","Type":"ContainerDied","Data":"6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564"}
Jul 30 23:36:49.828419 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.828380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr" event={"ID":"b7356341-92e7-424a-aa74-d52d63f23ea7","Type":"ContainerDied","Data":"a9a0b205641a9b4c20e556b46bfcaf2d3489ad140f4fdc8b909cf1dd4b61ce9b"}
Jul 30 23:36:49.828419 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.828339    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"
Jul 30 23:36:49.836551 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.836525    2578 scope.go:117] "RemoveContainer" containerID="1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc"
Jul 30 23:36:49.836901 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:36:49.836881    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc\": container with ID starting with 1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc not found: ID does not exist" containerID="1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc"
Jul 30 23:36:49.836962 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.836910    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc"} err="failed to get container status \"1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc\": rpc error: code = NotFound desc = could not find container \"1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc\": container with ID starting with 1fe67bd6afce53437bffafd8504b7974ea81c1c195094ebf762abf97a9fd4ccc not found: ID does not exist"
Jul 30 23:36:49.836962 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.836929    2578 scope.go:117] "RemoveContainer" containerID="6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564"
Jul 30 23:36:49.846344 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.846325    2578 scope.go:117] "RemoveContainer" containerID="6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564"
Jul 30 23:36:49.846634 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:36:49.846616    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564\": container with ID starting with 6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564 not found: ID does not exist" containerID="6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564"
Jul 30 23:36:49.846719 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.846643    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564"} err="failed to get container status \"6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564\": rpc error: code = NotFound desc = could not find container \"6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564\": container with ID starting with 6aa4222262ed33f76bcfbd579b5a0c441444a770cd543e310d5a85649a553564 not found: ID does not exist"
Jul 30 23:36:49.852319 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.852292    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"]
Jul 30 23:36:49.856273 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.856244    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-6a2f66-cb77c9f94-tkg4t"]
Jul 30 23:36:49.866628 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.866601    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"]
Jul 30 23:36:49.870233 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:49.870207    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6a2f66-7d574b7448-f4pcr"]
Jul 30 23:36:50.344403 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:50.344368    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8819248a-b2ef-439f-9197-581fcad3f278" path="/var/lib/kubelet/pods/8819248a-b2ef-439f-9197-581fcad3f278/volumes"
Jul 30 23:36:50.344850 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:50.344835    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7356341-92e7-424a-aa74-d52d63f23ea7" path="/var/lib/kubelet/pods/b7356341-92e7-424a-aa74-d52d63f23ea7/volumes"
Jul 30 23:36:59.215462 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.215427    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"]
Jul 30 23:36:59.215923 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.215830    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7356341-92e7-424a-aa74-d52d63f23ea7" containerName="payload-pre-processing"
Jul 30 23:36:59.215923 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.215843    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7356341-92e7-424a-aa74-d52d63f23ea7" containerName="payload-pre-processing"
Jul 30 23:36:59.215923 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.215868    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8819248a-b2ef-439f-9197-581fcad3f278" containerName="payload-processing"
Jul 30 23:36:59.215923 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.215874    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8819248a-b2ef-439f-9197-581fcad3f278" containerName="payload-processing"
Jul 30 23:36:59.215923 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.215924    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7356341-92e7-424a-aa74-d52d63f23ea7" containerName="payload-pre-processing"
Jul 30 23:36:59.216122 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.215934    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8819248a-b2ef-439f-9197-581fcad3f278" containerName="payload-processing"
Jul 30 23:36:59.220549 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.220528    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"
Jul 30 23:36:59.223377 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.223346    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jqcdc\""
Jul 30 23:36:59.225918 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.225890    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"]
Jul 30 23:36:59.276970 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.276923    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0be4b0-8e32-4fdc-9459-d11a11751346-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj\" (UID: \"6e0be4b0-8e32-4fdc-9459-d11a11751346\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"
Jul 30 23:36:59.377982 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.377938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0be4b0-8e32-4fdc-9459-d11a11751346-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj\" (UID: \"6e0be4b0-8e32-4fdc-9459-d11a11751346\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"
Jul 30 23:36:59.378545 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.378525    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0be4b0-8e32-4fdc-9459-d11a11751346-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj\" (UID: \"6e0be4b0-8e32-4fdc-9459-d11a11751346\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"
Jul 30 23:36:59.532275 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.532172    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"
Jul 30 23:36:59.671648 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.671523    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"]
Jul 30 23:36:59.674547 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:36:59.674507    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e0be4b0_8e32_4fdc_9459_d11a11751346.slice/crio-98a582632eb6f7c546385cb0dcfc002577f27eaa76ef5aa7f308cce559b49012 WatchSource:0}: Error finding container 98a582632eb6f7c546385cb0dcfc002577f27eaa76ef5aa7f308cce559b49012: Status 404 returned error can't find the container with id 98a582632eb6f7c546385cb0dcfc002577f27eaa76ef5aa7f308cce559b49012
Jul 30 23:36:59.883687 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:36:59.883656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj" event={"ID":"6e0be4b0-8e32-4fdc-9459-d11a11751346","Type":"ContainerStarted","Data":"98a582632eb6f7c546385cb0dcfc002577f27eaa76ef5aa7f308cce559b49012"}
Jul 30 23:37:00.890144 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:00.890109    2578 generic.go:356] "Generic (PLEG): container finished" podID="6e0be4b0-8e32-4fdc-9459-d11a11751346" containerID="ee04187086de0ae325965afbd783ee6c74973fdbcfc5764895d6b83518d042ab" exitCode=0
Jul 30 23:37:00.890546 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:00.890160    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj" event={"ID":"6e0be4b0-8e32-4fdc-9459-d11a11751346","Type":"ContainerDied","Data":"ee04187086de0ae325965afbd783ee6c74973fdbcfc5764895d6b83518d042ab"}
Jul 30 23:37:02.040563 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:02.040525    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"
Jul 30 23:37:02.100121 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:02.100070    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0be4b0-8e32-4fdc-9459-d11a11751346-maas-api-service-ca\") pod \"6e0be4b0-8e32-4fdc-9459-d11a11751346\" (UID: \"6e0be4b0-8e32-4fdc-9459-d11a11751346\") "
Jul 30 23:37:02.100429 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:02.100401    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e0be4b0-8e32-4fdc-9459-d11a11751346-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e0be4b0-8e32-4fdc-9459-d11a11751346" (UID: "6e0be4b0-8e32-4fdc-9459-d11a11751346"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:37:02.201083 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:02.201031    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e0be4b0-8e32-4fdc-9459-d11a11751346-maas-api-service-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:02.901124 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:02.901095    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"
Jul 30 23:37:02.901306 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:02.901094    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj" event={"ID":"6e0be4b0-8e32-4fdc-9459-d11a11751346","Type":"ContainerDied","Data":"98a582632eb6f7c546385cb0dcfc002577f27eaa76ef5aa7f308cce559b49012"}
Jul 30 23:37:02.901306 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:02.901219    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98a582632eb6f7c546385cb0dcfc002577f27eaa76ef5aa7f308cce559b49012"
Jul 30 23:37:14.693402 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.693362    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"]
Jul 30 23:37:14.693883 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.693599    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh" podUID="ecff7c9a-27d0-416a-88a6-fe64dfaef58f" containerName="payload-processing" containerID="cri-o://50fe2ea9957441f5da7f331363701ed84647ddb1335de76477653c132f444550" gracePeriod=30
Jul 30 23:37:14.702246 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.702211    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"]
Jul 30 23:37:14.702730 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.702696    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb" podUID="7c51e275-4b42-41db-aa3c-22f09ff9605a" containerName="payload-pre-processing" containerID="cri-o://bb8a46863baea5d3ae004f3db49813c0b7666b7d4205f3a149cc7a9c1ce79240" gracePeriod=30
Jul 30 23:37:14.834518 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.833669    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"]
Jul 30 23:37:14.836548 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.836518    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-287423-6gjrj"]
Jul 30 23:37:14.965731 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.965698    2578 generic.go:356] "Generic (PLEG): container finished" podID="ecff7c9a-27d0-416a-88a6-fe64dfaef58f" containerID="50fe2ea9957441f5da7f331363701ed84647ddb1335de76477653c132f444550" exitCode=0
Jul 30 23:37:14.965914 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.965784    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh" event={"ID":"ecff7c9a-27d0-416a-88a6-fe64dfaef58f","Type":"ContainerDied","Data":"50fe2ea9957441f5da7f331363701ed84647ddb1335de76477653c132f444550"}
Jul 30 23:37:14.965914 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.965855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh" event={"ID":"ecff7c9a-27d0-416a-88a6-fe64dfaef58f","Type":"ContainerDied","Data":"183f7a1f33f2a67fc687458c94997f605e5540df7c0e92242599e9b37e0a4e80"}
Jul 30 23:37:14.965914 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.965882    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="183f7a1f33f2a67fc687458c94997f605e5540df7c0e92242599e9b37e0a4e80"
Jul 30 23:37:14.967933 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.967903    2578 generic.go:356] "Generic (PLEG): container finished" podID="7c51e275-4b42-41db-aa3c-22f09ff9605a" containerID="bb8a46863baea5d3ae004f3db49813c0b7666b7d4205f3a149cc7a9c1ce79240" exitCode=0
Jul 30 23:37:14.968108 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.967994    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb" event={"ID":"7c51e275-4b42-41db-aa3c-22f09ff9605a","Type":"ContainerDied","Data":"bb8a46863baea5d3ae004f3db49813c0b7666b7d4205f3a149cc7a9c1ce79240"}
Jul 30 23:37:14.973184 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:14.973158    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:37:15.005031 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.005001    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:37:15.023077 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.023017    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-plugins-config-volume\") pod \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\" (UID: \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\") "
Jul 30 23:37:15.023668 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.023442    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-678px\" (UniqueName: \"kubernetes.io/projected/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-kube-api-access-678px\") pod \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\" (UID: \"ecff7c9a-27d0-416a-88a6-fe64dfaef58f\") "
Jul 30 23:37:15.023668 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.023624    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecff7c9a-27d0-416a-88a6-fe64dfaef58f" (UID: "ecff7c9a-27d0-416a-88a6-fe64dfaef58f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:37:15.023857 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.023759    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:15.026005 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.025978    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-kube-api-access-678px" (OuterVolumeSpecName: "kube-api-access-678px") pod "ecff7c9a-27d0-416a-88a6-fe64dfaef58f" (UID: "ecff7c9a-27d0-416a-88a6-fe64dfaef58f"). InnerVolumeSpecName "kube-api-access-678px". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:37:15.124674 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.124619    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c51e275-4b42-41db-aa3c-22f09ff9605a-plugins-config-volume\") pod \"7c51e275-4b42-41db-aa3c-22f09ff9605a\" (UID: \"7c51e275-4b42-41db-aa3c-22f09ff9605a\") "
Jul 30 23:37:15.124674 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.124680    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-679g4\" (UniqueName: \"kubernetes.io/projected/7c51e275-4b42-41db-aa3c-22f09ff9605a-kube-api-access-679g4\") pod \"7c51e275-4b42-41db-aa3c-22f09ff9605a\" (UID: \"7c51e275-4b42-41db-aa3c-22f09ff9605a\") "
Jul 30 23:37:15.124941 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.124923    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-678px\" (UniqueName: \"kubernetes.io/projected/ecff7c9a-27d0-416a-88a6-fe64dfaef58f-kube-api-access-678px\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:15.125108 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.125090    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c51e275-4b42-41db-aa3c-22f09ff9605a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c51e275-4b42-41db-aa3c-22f09ff9605a" (UID: "7c51e275-4b42-41db-aa3c-22f09ff9605a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:37:15.126931 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.126907    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c51e275-4b42-41db-aa3c-22f09ff9605a-kube-api-access-679g4" (OuterVolumeSpecName: "kube-api-access-679g4") pod "7c51e275-4b42-41db-aa3c-22f09ff9605a" (UID: "7c51e275-4b42-41db-aa3c-22f09ff9605a"). InnerVolumeSpecName "kube-api-access-679g4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:37:15.226489 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.226390    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c51e275-4b42-41db-aa3c-22f09ff9605a-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:15.226489 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.226423    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-679g4\" (UniqueName: \"kubernetes.io/projected/7c51e275-4b42-41db-aa3c-22f09ff9605a-kube-api-access-679g4\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:15.974092 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.973989    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"
Jul 30 23:37:15.974092 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.974007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb" event={"ID":"7c51e275-4b42-41db-aa3c-22f09ff9605a","Type":"ContainerDied","Data":"f38fd2ccc750e411f68a11b69a8ed09fa099ed7afccfebc64b6e4803e89404ec"}
Jul 30 23:37:15.974092 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.974084    2578 scope.go:117] "RemoveContainer" containerID="bb8a46863baea5d3ae004f3db49813c0b7666b7d4205f3a149cc7a9c1ce79240"
Jul 30 23:37:15.974092 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.973986    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"
Jul 30 23:37:15.998183 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:15.998140    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"]
Jul 30 23:37:16.001776 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:16.001740    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-287423-5f9659ffbd-c8kcb"]
Jul 30 23:37:16.011422 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:16.011392    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"]
Jul 30 23:37:16.015425 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:16.015391    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-287423-565b7d8c98-jbkmh"]
Jul 30 23:37:16.341981 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:16.341894    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e0be4b0-8e32-4fdc-9459-d11a11751346" path="/var/lib/kubelet/pods/6e0be4b0-8e32-4fdc-9459-d11a11751346/volumes"
Jul 30 23:37:16.342274 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:16.342261    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c51e275-4b42-41db-aa3c-22f09ff9605a" path="/var/lib/kubelet/pods/7c51e275-4b42-41db-aa3c-22f09ff9605a/volumes"
Jul 30 23:37:16.342567 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:16.342556    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecff7c9a-27d0-416a-88a6-fe64dfaef58f" path="/var/lib/kubelet/pods/ecff7c9a-27d0-416a-88a6-fe64dfaef58f/volumes"
Jul 30 23:37:20.206487 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:20.206453    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"]
Jul 30 23:37:35.226976 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.226924    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t" podUID="19837b0f-6578-4e47-adf9-f99c70e32afe" containerName="main" containerID="cri-o://4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee" gracePeriod=45
Jul 30 23:37:35.477317 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.477255    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:37:35.619230 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619182    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-model-cache\") pod \"19837b0f-6578-4e47-adf9-f99c70e32afe\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") "
Jul 30 23:37:35.619230 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619234    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdq58\" (UniqueName: \"kubernetes.io/projected/19837b0f-6578-4e47-adf9-f99c70e32afe-kube-api-access-bdq58\") pod \"19837b0f-6578-4e47-adf9-f99c70e32afe\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") "
Jul 30 23:37:35.619515 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619295    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-tmp-dir\") pod \"19837b0f-6578-4e47-adf9-f99c70e32afe\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") "
Jul 30 23:37:35.619515 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619324    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19837b0f-6578-4e47-adf9-f99c70e32afe-tls-certs\") pod \"19837b0f-6578-4e47-adf9-f99c70e32afe\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") "
Jul 30 23:37:35.619515 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619340    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-home\") pod \"19837b0f-6578-4e47-adf9-f99c70e32afe\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") "
Jul 30 23:37:35.619515 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619364    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-dshm\") pod \"19837b0f-6578-4e47-adf9-f99c70e32afe\" (UID: \"19837b0f-6578-4e47-adf9-f99c70e32afe\") "
Jul 30 23:37:35.619515 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619463    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-model-cache" (OuterVolumeSpecName: "model-cache") pod "19837b0f-6578-4e47-adf9-f99c70e32afe" (UID: "19837b0f-6578-4e47-adf9-f99c70e32afe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:37:35.619760 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619603    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19837b0f-6578-4e47-adf9-f99c70e32afe" (UID: "19837b0f-6578-4e47-adf9-f99c70e32afe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:37:35.619760 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619654    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-model-cache\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:35.619760 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.619657    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-home" (OuterVolumeSpecName: "home") pod "19837b0f-6578-4e47-adf9-f99c70e32afe" (UID: "19837b0f-6578-4e47-adf9-f99c70e32afe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:37:35.621804 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.621781    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-dshm" (OuterVolumeSpecName: "dshm") pod "19837b0f-6578-4e47-adf9-f99c70e32afe" (UID: "19837b0f-6578-4e47-adf9-f99c70e32afe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 30 23:37:35.621905 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.621802    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19837b0f-6578-4e47-adf9-f99c70e32afe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19837b0f-6578-4e47-adf9-f99c70e32afe" (UID: "19837b0f-6578-4e47-adf9-f99c70e32afe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 30 23:37:35.621948 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.621910    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19837b0f-6578-4e47-adf9-f99c70e32afe-kube-api-access-bdq58" (OuterVolumeSpecName: "kube-api-access-bdq58") pod "19837b0f-6578-4e47-adf9-f99c70e32afe" (UID: "19837b0f-6578-4e47-adf9-f99c70e32afe"). InnerVolumeSpecName "kube-api-access-bdq58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:37:35.720130 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.720097    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19837b0f-6578-4e47-adf9-f99c70e32afe-tls-certs\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:35.720130 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.720122    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-home\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:35.720130 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.720130    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-dshm\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:35.720130 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.720139    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdq58\" (UniqueName: \"kubernetes.io/projected/19837b0f-6578-4e47-adf9-f99c70e32afe-kube-api-access-bdq58\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:35.720429 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:35.720149    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19837b0f-6578-4e47-adf9-f99c70e32afe-tmp-dir\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:37:36.074073 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.074012    2578 generic.go:356] "Generic (PLEG): container finished" podID="19837b0f-6578-4e47-adf9-f99c70e32afe" containerID="4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee" exitCode=0
Jul 30 23:37:36.074240 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.074087    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"
Jul 30 23:37:36.074240 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.074098    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t" event={"ID":"19837b0f-6578-4e47-adf9-f99c70e32afe","Type":"ContainerDied","Data":"4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee"}
Jul 30 23:37:36.074240 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.074142    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t" event={"ID":"19837b0f-6578-4e47-adf9-f99c70e32afe","Type":"ContainerDied","Data":"e90308ccc769b87acf16ed67c52dfe437f0fbb28f071636ed13c79dc2702ba13"}
Jul 30 23:37:36.074240 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.074160    2578 scope.go:117] "RemoveContainer" containerID="4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee"
Jul 30 23:37:36.083464 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.083446    2578 scope.go:117] "RemoveContainer" containerID="4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee"
Jul 30 23:37:36.083743 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:37:36.083721    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee\": container with ID starting with 4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee not found: ID does not exist" containerID="4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee"
Jul 30 23:37:36.083790 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.083752    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee"} err="failed to get container status \"4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee\": rpc error: code = NotFound desc = could not find container \"4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee\": container with ID starting with 4d330a3d3bc04431e525f4770b295ecdaf77eddb3df20591726a0e3a288e9fee not found: ID does not exist"
Jul 30 23:37:36.094314 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.094291    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"]
Jul 30 23:37:36.100287 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.100263    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-287423/ipp-route-287423-kserve-57fd54d59-jw26t"]
Jul 30 23:37:36.342185 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:37:36.342102    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19837b0f-6578-4e47-adf9-f99c70e32afe" path="/var/lib/kubelet/pods/19837b0f-6578-4e47-adf9-f99c70e32afe/volumes"
Jul 30 23:38:17.733764 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.733724    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"]
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734109    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecff7c9a-27d0-416a-88a6-fe64dfaef58f" containerName="payload-processing"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734121    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecff7c9a-27d0-416a-88a6-fe64dfaef58f" containerName="payload-processing"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734138    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19837b0f-6578-4e47-adf9-f99c70e32afe" containerName="main"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734143    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="19837b0f-6578-4e47-adf9-f99c70e32afe" containerName="main"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734150    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c51e275-4b42-41db-aa3c-22f09ff9605a" containerName="payload-pre-processing"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734155    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c51e275-4b42-41db-aa3c-22f09ff9605a" containerName="payload-pre-processing"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734161    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e0be4b0-8e32-4fdc-9459-d11a11751346" containerName="revoke-keys"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734166    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e0be4b0-8e32-4fdc-9459-d11a11751346" containerName="revoke-keys"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734224    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecff7c9a-27d0-416a-88a6-fe64dfaef58f" containerName="payload-processing"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734235    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="19837b0f-6578-4e47-adf9-f99c70e32afe" containerName="main"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734240    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e0be4b0-8e32-4fdc-9459-d11a11751346" containerName="revoke-keys"
Jul 30 23:38:17.734264 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.734248    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c51e275-4b42-41db-aa3c-22f09ff9605a" containerName="payload-pre-processing"
Jul 30 23:38:17.736250 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.736233    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"
Jul 30 23:38:17.739430 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.739411    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jqcdc\""
Jul 30 23:38:17.747562 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.747528    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"]
Jul 30 23:38:17.869842 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.869808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc2258f-6c74-4ef0-a48f-3d3249e23ca6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf\" (UID: \"4bc2258f-6c74-4ef0-a48f-3d3249e23ca6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"
Jul 30 23:38:17.971197 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.971154    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc2258f-6c74-4ef0-a48f-3d3249e23ca6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf\" (UID: \"4bc2258f-6c74-4ef0-a48f-3d3249e23ca6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"
Jul 30 23:38:17.971799 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:17.971776    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc2258f-6c74-4ef0-a48f-3d3249e23ca6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf\" (UID: \"4bc2258f-6c74-4ef0-a48f-3d3249e23ca6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"
Jul 30 23:38:18.047847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:18.047747    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"
Jul 30 23:38:18.190871 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:18.190845    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"]
Jul 30 23:38:18.193401 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:38:18.193357    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc2258f_6c74_4ef0_a48f_3d3249e23ca6.slice/crio-52fb3deff50db4aba19a5a9b0b6048b08a470b3baeb07c7c63df52b4d56bba22 WatchSource:0}: Error finding container 52fb3deff50db4aba19a5a9b0b6048b08a470b3baeb07c7c63df52b4d56bba22: Status 404 returned error can't find the container with id 52fb3deff50db4aba19a5a9b0b6048b08a470b3baeb07c7c63df52b4d56bba22
Jul 30 23:38:18.285370 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:18.285334    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf" event={"ID":"4bc2258f-6c74-4ef0-a48f-3d3249e23ca6","Type":"ContainerStarted","Data":"52fb3deff50db4aba19a5a9b0b6048b08a470b3baeb07c7c63df52b4d56bba22"}
Jul 30 23:38:19.292147 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:19.292106    2578 generic.go:356] "Generic (PLEG): container finished" podID="4bc2258f-6c74-4ef0-a48f-3d3249e23ca6" containerID="33656d75d2c0432852f06ae990ccda8065e3a7b5042b7a546ab5711d37df4432" exitCode=0
Jul 30 23:38:19.292605 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:19.292234    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf" event={"ID":"4bc2258f-6c74-4ef0-a48f-3d3249e23ca6","Type":"ContainerDied","Data":"33656d75d2c0432852f06ae990ccda8065e3a7b5042b7a546ab5711d37df4432"}
Jul 30 23:38:20.443400 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:20.443371    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"
Jul 30 23:38:20.592435 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:20.592342    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc2258f-6c74-4ef0-a48f-3d3249e23ca6-maas-api-service-ca\") pod \"4bc2258f-6c74-4ef0-a48f-3d3249e23ca6\" (UID: \"4bc2258f-6c74-4ef0-a48f-3d3249e23ca6\") "
Jul 30 23:38:20.592684 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:20.592660    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bc2258f-6c74-4ef0-a48f-3d3249e23ca6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4bc2258f-6c74-4ef0-a48f-3d3249e23ca6" (UID: "4bc2258f-6c74-4ef0-a48f-3d3249e23ca6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:38:20.693327 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:20.693289    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc2258f-6c74-4ef0-a48f-3d3249e23ca6-maas-api-service-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:21.308545 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:21.308515    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"
Jul 30 23:38:21.308545 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:21.308522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf" event={"ID":"4bc2258f-6c74-4ef0-a48f-3d3249e23ca6","Type":"ContainerDied","Data":"52fb3deff50db4aba19a5a9b0b6048b08a470b3baeb07c7c63df52b4d56bba22"}
Jul 30 23:38:21.308821 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:21.308568    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52fb3deff50db4aba19a5a9b0b6048b08a470b3baeb07c7c63df52b4d56bba22"
Jul 30 23:38:28.158954 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.158906    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"]
Jul 30 23:38:28.159535 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.159229    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g" podUID="5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e" containerName="payload-processing" containerID="cri-o://8d660f229eb0ba3ed74f8afd8c9e1b4bf4009af6636876599718f3aac8395e5e" gracePeriod=30
Jul 30 23:38:28.167626 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.167324    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"]
Jul 30 23:38:28.167626 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.167564    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2" podUID="4ab6814c-58eb-4115-9375-105fa30afde9" containerName="payload-pre-processing" containerID="cri-o://53bfe861ac9ae92f6c62d34e866fe99aa5c1c429347ce1ba34b6df120ae1e9df" gracePeriod=30
Jul 30 23:38:28.296585 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.296552    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"]
Jul 30 23:38:28.299237 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.299210    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-02337c-xnscf"]
Jul 30 23:38:28.346725 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.346683    2578 generic.go:356] "Generic (PLEG): container finished" podID="4ab6814c-58eb-4115-9375-105fa30afde9" containerID="53bfe861ac9ae92f6c62d34e866fe99aa5c1c429347ce1ba34b6df120ae1e9df" exitCode=0
Jul 30 23:38:28.348282 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.348255    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bc2258f-6c74-4ef0-a48f-3d3249e23ca6" path="/var/lib/kubelet/pods/4bc2258f-6c74-4ef0-a48f-3d3249e23ca6/volumes"
Jul 30 23:38:28.348584 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.348563    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2" event={"ID":"4ab6814c-58eb-4115-9375-105fa30afde9","Type":"ContainerDied","Data":"53bfe861ac9ae92f6c62d34e866fe99aa5c1c429347ce1ba34b6df120ae1e9df"}
Jul 30 23:38:28.348787 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.348766    2578 generic.go:356] "Generic (PLEG): container finished" podID="5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e" containerID="8d660f229eb0ba3ed74f8afd8c9e1b4bf4009af6636876599718f3aac8395e5e" exitCode=0
Jul 30 23:38:28.348855 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.348825    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g" event={"ID":"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e","Type":"ContainerDied","Data":"8d660f229eb0ba3ed74f8afd8c9e1b4bf4009af6636876599718f3aac8395e5e"}
Jul 30 23:38:28.451126 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.451101    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:38:28.479570 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.479538    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:38:28.565873 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.565841    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcmv6\" (UniqueName: \"kubernetes.io/projected/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-kube-api-access-xcmv6\") pod \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\" (UID: \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\") "
Jul 30 23:38:28.566101 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.565915    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-plugins-config-volume\") pod \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\" (UID: \"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e\") "
Jul 30 23:38:28.566101 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.565942    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bth8g\" (UniqueName: \"kubernetes.io/projected/4ab6814c-58eb-4115-9375-105fa30afde9-kube-api-access-bth8g\") pod \"4ab6814c-58eb-4115-9375-105fa30afde9\" (UID: \"4ab6814c-58eb-4115-9375-105fa30afde9\") "
Jul 30 23:38:28.566101 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.566038    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ab6814c-58eb-4115-9375-105fa30afde9-plugins-config-volume\") pod \"4ab6814c-58eb-4115-9375-105fa30afde9\" (UID: \"4ab6814c-58eb-4115-9375-105fa30afde9\") "
Jul 30 23:38:28.566377 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.566350    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e" (UID: "5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:38:28.566377 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.566368    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ab6814c-58eb-4115-9375-105fa30afde9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ab6814c-58eb-4115-9375-105fa30afde9" (UID: "4ab6814c-58eb-4115-9375-105fa30afde9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:38:28.568208 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.568183    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-kube-api-access-xcmv6" (OuterVolumeSpecName: "kube-api-access-xcmv6") pod "5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e" (UID: "5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e"). InnerVolumeSpecName "kube-api-access-xcmv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:38:28.568283 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.568227    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab6814c-58eb-4115-9375-105fa30afde9-kube-api-access-bth8g" (OuterVolumeSpecName: "kube-api-access-bth8g") pod "4ab6814c-58eb-4115-9375-105fa30afde9" (UID: "4ab6814c-58eb-4115-9375-105fa30afde9"). InnerVolumeSpecName "kube-api-access-bth8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:38:28.667742 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.667708    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcmv6\" (UniqueName: \"kubernetes.io/projected/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-kube-api-access-xcmv6\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:28.667742 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.667739    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:28.667742 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.667748    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bth8g\" (UniqueName: \"kubernetes.io/projected/4ab6814c-58eb-4115-9375-105fa30afde9-kube-api-access-bth8g\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:28.667968 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:28.667758    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ab6814c-58eb-4115-9375-105fa30afde9-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:29.355235 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.355205    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"
Jul 30 23:38:29.355727 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.355250    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g" event={"ID":"5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e","Type":"ContainerDied","Data":"52534018ba3a249ee879d3e39a6382523fbabe1aface6ff83d19b34f54eafc3f"}
Jul 30 23:38:29.355727 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.355317    2578 scope.go:117] "RemoveContainer" containerID="8d660f229eb0ba3ed74f8afd8c9e1b4bf4009af6636876599718f3aac8395e5e"
Jul 30 23:38:29.356730 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.356706    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2" event={"ID":"4ab6814c-58eb-4115-9375-105fa30afde9","Type":"ContainerDied","Data":"f91222276b558c0fd47f9e9cd064501372ffd4eeb748dfec584086e1180ad9a3"}
Jul 30 23:38:29.356806 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.356744    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"
Jul 30 23:38:29.365429 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.365409    2578 scope.go:117] "RemoveContainer" containerID="53bfe861ac9ae92f6c62d34e866fe99aa5c1c429347ce1ba34b6df120ae1e9df"
Jul 30 23:38:29.379719 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.379694    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"]
Jul 30 23:38:29.383725 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.383700    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-02337c-5bdf8d6444-jpr4g"]
Jul 30 23:38:29.393813 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.393784    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"]
Jul 30 23:38:29.396844 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:29.396816    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-02337c-5d556d4f9b-hr5l2"]
Jul 30 23:38:30.342169 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:30.342134    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ab6814c-58eb-4115-9375-105fa30afde9" path="/var/lib/kubelet/pods/4ab6814c-58eb-4115-9375-105fa30afde9/volumes"
Jul 30 23:38:30.342644 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:30.342626    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e" path="/var/lib/kubelet/pods/5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e/volumes"
Jul 30 23:38:35.181394 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181358    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"]
Jul 30 23:38:35.181777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181700    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ab6814c-58eb-4115-9375-105fa30afde9" containerName="payload-pre-processing"
Jul 30 23:38:35.181777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181711    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ab6814c-58eb-4115-9375-105fa30afde9" containerName="payload-pre-processing"
Jul 30 23:38:35.181777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181722    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e" containerName="payload-processing"
Jul 30 23:38:35.181777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181728    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e" containerName="payload-processing"
Jul 30 23:38:35.181777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181738    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc2258f-6c74-4ef0-a48f-3d3249e23ca6" containerName="revoke-keys"
Jul 30 23:38:35.181777 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181743    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc2258f-6c74-4ef0-a48f-3d3249e23ca6" containerName="revoke-keys"
Jul 30 23:38:35.181966 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181808    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b9ac5f7-d615-4ce1-88d9-48c3bec0e54e" containerName="payload-processing"
Jul 30 23:38:35.181966 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181815    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bc2258f-6c74-4ef0-a48f-3d3249e23ca6" containerName="revoke-keys"
Jul 30 23:38:35.181966 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.181821    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ab6814c-58eb-4115-9375-105fa30afde9" containerName="payload-pre-processing"
Jul 30 23:38:35.183717 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.183701    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"
Jul 30 23:38:35.185969 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.185950    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jqcdc\""
Jul 30 23:38:35.191328 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.191303    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"]
Jul 30 23:38:35.331728 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.331688    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ad85a0-7ff0-49cf-9172-d3209f318fde-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-80952f-tpprh\" (UID: \"96ad85a0-7ff0-49cf-9172-d3209f318fde\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"
Jul 30 23:38:35.433171 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.433082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ad85a0-7ff0-49cf-9172-d3209f318fde-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-80952f-tpprh\" (UID: \"96ad85a0-7ff0-49cf-9172-d3209f318fde\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"
Jul 30 23:38:35.433769 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.433744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ad85a0-7ff0-49cf-9172-d3209f318fde-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-80952f-tpprh\" (UID: \"96ad85a0-7ff0-49cf-9172-d3209f318fde\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"
Jul 30 23:38:35.494898 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.494868    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"
Jul 30 23:38:35.631373 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:35.631320    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"]
Jul 30 23:38:35.633595 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:38:35.633560    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ad85a0_7ff0_49cf_9172_d3209f318fde.slice/crio-93b58c65f5c968d6bfae6a24cd7705eff9e13b19b8b651d29c5562e56384ccc2 WatchSource:0}: Error finding container 93b58c65f5c968d6bfae6a24cd7705eff9e13b19b8b651d29c5562e56384ccc2: Status 404 returned error can't find the container with id 93b58c65f5c968d6bfae6a24cd7705eff9e13b19b8b651d29c5562e56384ccc2
Jul 30 23:38:36.394522 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:36.394435    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh" event={"ID":"96ad85a0-7ff0-49cf-9172-d3209f318fde","Type":"ContainerStarted","Data":"afb40ddc9a278b999d362003c47156d6aee99b2335c656d99830d3e84474bdc6"}
Jul 30 23:38:36.394522 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:36.394500    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh" event={"ID":"96ad85a0-7ff0-49cf-9172-d3209f318fde","Type":"ContainerStarted","Data":"93b58c65f5c968d6bfae6a24cd7705eff9e13b19b8b651d29c5562e56384ccc2"}
Jul 30 23:38:36.417172 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:36.417121    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh" podStartSLOduration=1.417104934 podStartE2EDuration="1.417104934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-30 23:38:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:38:36.416261622 +0000 UTC m=+3496.591225301" watchObservedRunningTime="2026-07-30 23:38:36.417104934 +0000 UTC m=+3496.592068657"
Jul 30 23:38:37.403428 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:37.403386    2578 generic.go:356] "Generic (PLEG): container finished" podID="96ad85a0-7ff0-49cf-9172-d3209f318fde" containerID="afb40ddc9a278b999d362003c47156d6aee99b2335c656d99830d3e84474bdc6" exitCode=0
Jul 30 23:38:37.403829 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:37.403459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh" event={"ID":"96ad85a0-7ff0-49cf-9172-d3209f318fde","Type":"ContainerDied","Data":"afb40ddc9a278b999d362003c47156d6aee99b2335c656d99830d3e84474bdc6"}
Jul 30 23:38:38.544350 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:38.544325    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"
Jul 30 23:38:38.562806 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:38.562773    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ad85a0-7ff0-49cf-9172-d3209f318fde-maas-api-service-ca\") pod \"96ad85a0-7ff0-49cf-9172-d3209f318fde\" (UID: \"96ad85a0-7ff0-49cf-9172-d3209f318fde\") "
Jul 30 23:38:38.563163 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:38.563140    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96ad85a0-7ff0-49cf-9172-d3209f318fde-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96ad85a0-7ff0-49cf-9172-d3209f318fde" (UID: "96ad85a0-7ff0-49cf-9172-d3209f318fde"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:38:38.663811 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:38.663725    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96ad85a0-7ff0-49cf-9172-d3209f318fde-maas-api-service-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:39.413439 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:39.413408    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"
Jul 30 23:38:39.413613 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:39.413409    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh" event={"ID":"96ad85a0-7ff0-49cf-9172-d3209f318fde","Type":"ContainerDied","Data":"93b58c65f5c968d6bfae6a24cd7705eff9e13b19b8b651d29c5562e56384ccc2"}
Jul 30 23:38:39.413613 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:39.413540    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93b58c65f5c968d6bfae6a24cd7705eff9e13b19b8b651d29c5562e56384ccc2"
Jul 30 23:38:45.793424 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:45.793344    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"]
Jul 30 23:38:45.793892 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:45.793632    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q" podUID="c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca" containerName="payload-processing" containerID="cri-o://fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381" gracePeriod=30
Jul 30 23:38:45.800193 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:45.799975    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"]
Jul 30 23:38:45.800359 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:45.800255    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp" podUID="6b166424-9e71-4dd5-9617-f716975dfa77" containerName="payload-pre-processing" containerID="cri-o://9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54" gracePeriod=30
Jul 30 23:38:45.930847 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:45.930761    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"]
Jul 30 23:38:45.932554 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:45.932525    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-80952f-tpprh"]
Jul 30 23:38:46.059864 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.059839    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:38:46.086281 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.086257    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:38:46.128175 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.128145    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b166424-9e71-4dd5-9617-f716975dfa77-plugins-config-volume\") pod \"6b166424-9e71-4dd5-9617-f716975dfa77\" (UID: \"6b166424-9e71-4dd5-9617-f716975dfa77\") "
Jul 30 23:38:46.128338 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.128228    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t68br\" (UniqueName: \"kubernetes.io/projected/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-kube-api-access-t68br\") pod \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\" (UID: \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\") "
Jul 30 23:38:46.128338 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.128257    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r9b6\" (UniqueName: \"kubernetes.io/projected/6b166424-9e71-4dd5-9617-f716975dfa77-kube-api-access-2r9b6\") pod \"6b166424-9e71-4dd5-9617-f716975dfa77\" (UID: \"6b166424-9e71-4dd5-9617-f716975dfa77\") "
Jul 30 23:38:46.128338 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.128307    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-plugins-config-volume\") pod \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\" (UID: \"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca\") "
Jul 30 23:38:46.128534 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.128509    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b166424-9e71-4dd5-9617-f716975dfa77-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b166424-9e71-4dd5-9617-f716975dfa77" (UID: "6b166424-9e71-4dd5-9617-f716975dfa77"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:38:46.128726 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.128705    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca" (UID: "c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:38:46.130415 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.130394    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-kube-api-access-t68br" (OuterVolumeSpecName: "kube-api-access-t68br") pod "c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca" (UID: "c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca"). InnerVolumeSpecName "kube-api-access-t68br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:38:46.130484 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.130460    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b166424-9e71-4dd5-9617-f716975dfa77-kube-api-access-2r9b6" (OuterVolumeSpecName: "kube-api-access-2r9b6") pod "6b166424-9e71-4dd5-9617-f716975dfa77" (UID: "6b166424-9e71-4dd5-9617-f716975dfa77"). InnerVolumeSpecName "kube-api-access-2r9b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:38:46.229129 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.229093    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b166424-9e71-4dd5-9617-f716975dfa77-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:46.229129 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.229121    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t68br\" (UniqueName: \"kubernetes.io/projected/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-kube-api-access-t68br\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:46.229129 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.229134    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r9b6\" (UniqueName: \"kubernetes.io/projected/6b166424-9e71-4dd5-9617-f716975dfa77-kube-api-access-2r9b6\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:46.229365 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.229145    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:46.342187 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.342096    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96ad85a0-7ff0-49cf-9172-d3209f318fde" path="/var/lib/kubelet/pods/96ad85a0-7ff0-49cf-9172-d3209f318fde/volumes"
Jul 30 23:38:46.449817 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.449775    2578 generic.go:356] "Generic (PLEG): container finished" podID="c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca" containerID="fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381" exitCode=0
Jul 30 23:38:46.450014 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.449830    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"
Jul 30 23:38:46.450014 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.449874    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q" event={"ID":"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca","Type":"ContainerDied","Data":"fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381"}
Jul 30 23:38:46.450014 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.449938    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q" event={"ID":"c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca","Type":"ContainerDied","Data":"ed8e620edf8451e20171643677a169840769a091ed41ff8515982aa58f755d99"}
Jul 30 23:38:46.450014 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.449972    2578 scope.go:117] "RemoveContainer" containerID="fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381"
Jul 30 23:38:46.451359 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.451332    2578 generic.go:356] "Generic (PLEG): container finished" podID="6b166424-9e71-4dd5-9617-f716975dfa77" containerID="9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54" exitCode=0
Jul 30 23:38:46.451505 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.451387    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"
Jul 30 23:38:46.451505 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.451399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp" event={"ID":"6b166424-9e71-4dd5-9617-f716975dfa77","Type":"ContainerDied","Data":"9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54"}
Jul 30 23:38:46.451505 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.451433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp" event={"ID":"6b166424-9e71-4dd5-9617-f716975dfa77","Type":"ContainerDied","Data":"0f21c75dd3f771569740d7a6efb27cfc43b66328d7c0bf4778769d25517e91cb"}
Jul 30 23:38:46.459542 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.459523    2578 scope.go:117] "RemoveContainer" containerID="fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381"
Jul 30 23:38:46.459899 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:38:46.459880    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381\": container with ID starting with fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381 not found: ID does not exist" containerID="fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381"
Jul 30 23:38:46.459962 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.459908    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381"} err="failed to get container status \"fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381\": rpc error: code = NotFound desc = could not find container \"fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381\": container with ID starting with fdec5bf821e047468c9eb2cd5fe57202f74872fc31e3898b92aa483987817381 not found: ID does not exist"
Jul 30 23:38:46.459962 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.459925    2578 scope.go:117] "RemoveContainer" containerID="9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54"
Jul 30 23:38:46.467896 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.467705    2578 scope.go:117] "RemoveContainer" containerID="9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54"
Jul 30 23:38:46.468038 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:38:46.468013    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54\": container with ID starting with 9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54 not found: ID does not exist" containerID="9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54"
Jul 30 23:38:46.468152 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.468059    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54"} err="failed to get container status \"9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54\": rpc error: code = NotFound desc = could not find container \"9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54\": container with ID starting with 9125b48e917e7d1e4f1495a472da6de0ab60e3bcccc8ed20d367f1aedb4f3a54 not found: ID does not exist"
Jul 30 23:38:46.469435 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.469417    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"]
Jul 30 23:38:46.473600 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.473580    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-80952f-68f47c47bb-tj66q"]
Jul 30 23:38:46.483479 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.483455    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"]
Jul 30 23:38:46.487488 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:46.487469    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-80952f-6945854cfb-bbsjp"]
Jul 30 23:38:48.342375 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:48.342340    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b166424-9e71-4dd5-9617-f716975dfa77" path="/var/lib/kubelet/pods/6b166424-9e71-4dd5-9617-f716975dfa77/volumes"
Jul 30 23:38:48.342736 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:48.342678    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca" path="/var/lib/kubelet/pods/c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca/volumes"
Jul 30 23:38:53.206452 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206418    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"]
Jul 30 23:38:53.206853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206738    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ad85a0-7ff0-49cf-9172-d3209f318fde" containerName="revoke-keys"
Jul 30 23:38:53.206853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206749    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ad85a0-7ff0-49cf-9172-d3209f318fde" containerName="revoke-keys"
Jul 30 23:38:53.206853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206758    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca" containerName="payload-processing"
Jul 30 23:38:53.206853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206764    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca" containerName="payload-processing"
Jul 30 23:38:53.206853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206783    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b166424-9e71-4dd5-9617-f716975dfa77" containerName="payload-pre-processing"
Jul 30 23:38:53.206853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206789    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b166424-9e71-4dd5-9617-f716975dfa77" containerName="payload-pre-processing"
Jul 30 23:38:53.206853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206841    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b166424-9e71-4dd5-9617-f716975dfa77" containerName="payload-pre-processing"
Jul 30 23:38:53.206853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206853    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ad85a0-7ff0-49cf-9172-d3209f318fde" containerName="revoke-keys"
Jul 30 23:38:53.206853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.206858    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7340ccf-838e-4a29-9ed0-5c0cf6a5bcca" containerName="payload-processing"
Jul 30 23:38:53.209580 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.209563    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"
Jul 30 23:38:53.212127 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.212105    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jqcdc\""
Jul 30 23:38:53.220126 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.220092    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"]
Jul 30 23:38:53.291306 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.291267    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6996cce5-35f4-4589-a6c5-7cf81ebab24d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6\" (UID: \"6996cce5-35f4-4589-a6c5-7cf81ebab24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"
Jul 30 23:38:53.392764 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.392725    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6996cce5-35f4-4589-a6c5-7cf81ebab24d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6\" (UID: \"6996cce5-35f4-4589-a6c5-7cf81ebab24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"
Jul 30 23:38:53.393384 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.393363    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6996cce5-35f4-4589-a6c5-7cf81ebab24d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6\" (UID: \"6996cce5-35f4-4589-a6c5-7cf81ebab24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"
Jul 30 23:38:53.526934 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.526849    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"
Jul 30 23:38:53.661156 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:53.661128    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"]
Jul 30 23:38:53.663896 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:38:53.663863    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6996cce5_35f4_4589_a6c5_7cf81ebab24d.slice/crio-50acb0ff0b5e910896027688b7e3ee837f51b02a1c0df0a09479ed64de6ca987 WatchSource:0}: Error finding container 50acb0ff0b5e910896027688b7e3ee837f51b02a1c0df0a09479ed64de6ca987: Status 404 returned error can't find the container with id 50acb0ff0b5e910896027688b7e3ee837f51b02a1c0df0a09479ed64de6ca987
Jul 30 23:38:54.492853 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:54.492815    2578 generic.go:356] "Generic (PLEG): container finished" podID="6996cce5-35f4-4589-a6c5-7cf81ebab24d" containerID="0bd2ce382ad23278fcff54f809e04449ca6054b4511c851ca29694079536c505" exitCode=0
Jul 30 23:38:54.493261 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:54.492901    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6" event={"ID":"6996cce5-35f4-4589-a6c5-7cf81ebab24d","Type":"ContainerDied","Data":"0bd2ce382ad23278fcff54f809e04449ca6054b4511c851ca29694079536c505"}
Jul 30 23:38:54.493261 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:54.492955    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6" event={"ID":"6996cce5-35f4-4589-a6c5-7cf81ebab24d","Type":"ContainerStarted","Data":"50acb0ff0b5e910896027688b7e3ee837f51b02a1c0df0a09479ed64de6ca987"}
Jul 30 23:38:55.631021 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:55.630998    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"
Jul 30 23:38:55.713142 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:55.713103    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6996cce5-35f4-4589-a6c5-7cf81ebab24d-maas-api-service-ca\") pod \"6996cce5-35f4-4589-a6c5-7cf81ebab24d\" (UID: \"6996cce5-35f4-4589-a6c5-7cf81ebab24d\") "
Jul 30 23:38:55.713472 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:55.713448    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6996cce5-35f4-4589-a6c5-7cf81ebab24d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6996cce5-35f4-4589-a6c5-7cf81ebab24d" (UID: "6996cce5-35f4-4589-a6c5-7cf81ebab24d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:38:55.814639 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:55.814547    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6996cce5-35f4-4589-a6c5-7cf81ebab24d-maas-api-service-ca\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:38:56.503834 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:56.503803    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"
Jul 30 23:38:56.503834 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:56.503821    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6" event={"ID":"6996cce5-35f4-4589-a6c5-7cf81ebab24d","Type":"ContainerDied","Data":"50acb0ff0b5e910896027688b7e3ee837f51b02a1c0df0a09479ed64de6ca987"}
Jul 30 23:38:56.504134 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:38:56.503864    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50acb0ff0b5e910896027688b7e3ee837f51b02a1c0df0a09479ed64de6ca987"
Jul 30 23:39:03.648715 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.648671    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"]
Jul 30 23:39:03.649313 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.648935    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42" podUID="14abd2ab-cd5d-4848-b181-e04c5a9c281e" containerName="payload-processing" containerID="cri-o://7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416" gracePeriod=30
Jul 30 23:39:03.655498 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.655452    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"]
Jul 30 23:39:03.655738 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.655707    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv" podUID="6700b22b-3a64-42da-aa0a-86820218d15e" containerName="payload-pre-processing" containerID="cri-o://f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0" gracePeriod=30
Jul 30 23:39:03.783723 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.783673    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"]
Jul 30 23:39:03.786736 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.786687    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb1098-mswv6"]
Jul 30 23:39:03.912750 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.912683    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:39:03.944045 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.944014    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:39:03.984748 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.984710    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14abd2ab-cd5d-4848-b181-e04c5a9c281e-plugins-config-volume\") pod \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\" (UID: \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\") "
Jul 30 23:39:03.984748 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.984758    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gcw7p\" (UniqueName: \"kubernetes.io/projected/14abd2ab-cd5d-4848-b181-e04c5a9c281e-kube-api-access-gcw7p\") pod \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\" (UID: \"14abd2ab-cd5d-4848-b181-e04c5a9c281e\") "
Jul 30 23:39:03.985186 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.985159    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14abd2ab-cd5d-4848-b181-e04c5a9c281e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "14abd2ab-cd5d-4848-b181-e04c5a9c281e" (UID: "14abd2ab-cd5d-4848-b181-e04c5a9c281e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:39:03.987023 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:03.987000    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14abd2ab-cd5d-4848-b181-e04c5a9c281e-kube-api-access-gcw7p" (OuterVolumeSpecName: "kube-api-access-gcw7p") pod "14abd2ab-cd5d-4848-b181-e04c5a9c281e" (UID: "14abd2ab-cd5d-4848-b181-e04c5a9c281e"). InnerVolumeSpecName "kube-api-access-gcw7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:39:04.085676 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.085630    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6700b22b-3a64-42da-aa0a-86820218d15e-plugins-config-volume\") pod \"6700b22b-3a64-42da-aa0a-86820218d15e\" (UID: \"6700b22b-3a64-42da-aa0a-86820218d15e\") "
Jul 30 23:39:04.085676 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.085690    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l88rv\" (UniqueName: \"kubernetes.io/projected/6700b22b-3a64-42da-aa0a-86820218d15e-kube-api-access-l88rv\") pod \"6700b22b-3a64-42da-aa0a-86820218d15e\" (UID: \"6700b22b-3a64-42da-aa0a-86820218d15e\") "
Jul 30 23:39:04.085944 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.085836    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14abd2ab-cd5d-4848-b181-e04c5a9c281e-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:39:04.085944 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.085845    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcw7p\" (UniqueName: \"kubernetes.io/projected/14abd2ab-cd5d-4848-b181-e04c5a9c281e-kube-api-access-gcw7p\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:39:04.086100 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.086022    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6700b22b-3a64-42da-aa0a-86820218d15e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6700b22b-3a64-42da-aa0a-86820218d15e" (UID: "6700b22b-3a64-42da-aa0a-86820218d15e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 30 23:39:04.087990 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.087963    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6700b22b-3a64-42da-aa0a-86820218d15e-kube-api-access-l88rv" (OuterVolumeSpecName: "kube-api-access-l88rv") pod "6700b22b-3a64-42da-aa0a-86820218d15e" (UID: "6700b22b-3a64-42da-aa0a-86820218d15e"). InnerVolumeSpecName "kube-api-access-l88rv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 30 23:39:04.186820 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.186719    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6700b22b-3a64-42da-aa0a-86820218d15e-plugins-config-volume\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:39:04.186820 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.186754    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l88rv\" (UniqueName: \"kubernetes.io/projected/6700b22b-3a64-42da-aa0a-86820218d15e-kube-api-access-l88rv\") on node \"ip-10-0-142-225.ec2.internal\" DevicePath \"\""
Jul 30 23:39:04.343739 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.343690    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6996cce5-35f4-4589-a6c5-7cf81ebab24d" path="/var/lib/kubelet/pods/6996cce5-35f4-4589-a6c5-7cf81ebab24d/volumes"
Jul 30 23:39:04.546616 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.546581    2578 generic.go:356] "Generic (PLEG): container finished" podID="14abd2ab-cd5d-4848-b181-e04c5a9c281e" containerID="7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416" exitCode=0
Jul 30 23:39:04.546811 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.546622    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"
Jul 30 23:39:04.546811 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.546668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42" event={"ID":"14abd2ab-cd5d-4848-b181-e04c5a9c281e","Type":"ContainerDied","Data":"7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416"}
Jul 30 23:39:04.546811 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.546720    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42" event={"ID":"14abd2ab-cd5d-4848-b181-e04c5a9c281e","Type":"ContainerDied","Data":"7a9e42a21bd8c1a2a662ba3e2349997bb0592bcbe3e16d659c60c18c6fca59f1"}
Jul 30 23:39:04.546811 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.546750    2578 scope.go:117] "RemoveContainer" containerID="7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416"
Jul 30 23:39:04.548190 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.548114    2578 generic.go:356] "Generic (PLEG): container finished" podID="6700b22b-3a64-42da-aa0a-86820218d15e" containerID="f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0" exitCode=0
Jul 30 23:39:04.548190 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.548166    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"
Jul 30 23:39:04.548348 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.548194    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv" event={"ID":"6700b22b-3a64-42da-aa0a-86820218d15e","Type":"ContainerDied","Data":"f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0"}
Jul 30 23:39:04.548348 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.548238    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv" event={"ID":"6700b22b-3a64-42da-aa0a-86820218d15e","Type":"ContainerDied","Data":"08f8deed7a8d7c05cbd24ccf76627a161effbe1246212f677cccd3df298a46af"}
Jul 30 23:39:04.557196 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.557172    2578 scope.go:117] "RemoveContainer" containerID="7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416"
Jul 30 23:39:04.557492 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:39:04.557472    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416\": container with ID starting with 7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416 not found: ID does not exist" containerID="7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416"
Jul 30 23:39:04.557570 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.557502    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416"} err="failed to get container status \"7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416\": rpc error: code = NotFound desc = could not find container \"7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416\": container with ID starting with 7c186470021cd9f60b42fafda273ee222c1a84d35da4bd30ccf1eb72b7b9b416 not found: ID does not exist"
Jul 30 23:39:04.557570 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.557520    2578 scope.go:117] "RemoveContainer" containerID="f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0"
Jul 30 23:39:04.565814 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.565784    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"]
Jul 30 23:39:04.566311 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.566291    2578 scope.go:117] "RemoveContainer" containerID="f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0"
Jul 30 23:39:04.566647 ip-10-0-142-225 kubenswrapper[2578]: E0730 23:39:04.566628    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0\": container with ID starting with f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0 not found: ID does not exist" containerID="f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0"
Jul 30 23:39:04.566698 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.566656    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0"} err="failed to get container status \"f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0\": rpc error: code = NotFound desc = could not find container \"f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0\": container with ID starting with f4d081ef4162760a5d406fe619f620c77805e9a66f79bbd632f898d4f08881f0 not found: ID does not exist"
Jul 30 23:39:04.573467 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.573420    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-bb1098-757ddc4556-5b2mv"]
Jul 30 23:39:04.594592 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.594560    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"]
Jul 30 23:39:04.603454 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:04.603425    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-bb1098-7597dccb54-hhh42"]
Jul 30 23:39:06.341572 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:06.341537    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14abd2ab-cd5d-4848-b181-e04c5a9c281e" path="/var/lib/kubelet/pods/14abd2ab-cd5d-4848-b181-e04c5a9c281e/volumes"
Jul 30 23:39:06.342088 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:06.341874    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6700b22b-3a64-42da-aa0a-86820218d15e" path="/var/lib/kubelet/pods/6700b22b-3a64-42da-aa0a-86820218d15e/volumes"
Jul 30 23:39:16.853722 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:16.853692    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wkz9w_7130dbaf-62b7-4d36-bbfb-a031aaa8f782/manager/0.log"
Jul 30 23:39:16.978946 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:16.978917    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jjs7h_1780e2f1-70d4-4b00-9a5f-595c19eaee9d/manager/0.log"
Jul 30 23:39:17.344917 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:17.344883    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5w7xx_b65d94e7-4b50-41ec-9c55-cfda5fc159c3/manager/0.log"
Jul 30 23:39:17.603852 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:17.603777    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff8c5f475-ghbrv_b671cb7b-cf47-4a48-b297-bd65c9b46755/copy-manifests/0.log"
Jul 30 23:39:17.611152 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:17.611118    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff8c5f475-ghbrv_b671cb7b-cf47-4a48-b297-bd65c9b46755/manager/0.log"
Jul 30 23:39:18.592598 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:18.592568    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-skrkb_9dcdc4b6-ce5d-496f-af33-2a615f771ae4/postgres/0.log"
Jul 30 23:39:19.553312 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:19.553278    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-qpwsq_091be627-208c-484a-966b-f56076624fb4/manager/0.log"
Jul 30 23:39:19.675713 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:19.675683    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-6d7qw_c2deeb6e-cd26-4ffd-9eed-f830be5d6c77/manager/0.log"
Jul 30 23:39:19.797893 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:19.797863    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-xwqsq_80b354f0-4267-46c8-a78c-86f2fcb5425f/kuadrant-console-plugin/0.log"
Jul 30 23:39:19.921413 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:19.921345    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-q2hm7_06fd676b-515b-40a7-9205-a0623478c2ba/manager/0.log"
Jul 30 23:39:20.151798 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:20.151770    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-nfs8s_0c8e620f-9826-4761-9ab2-e99a790ce61e/manager/0.log"
Jul 30 23:39:20.861818 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:20.861787    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b9c58d584-x4tth_f0ff893d-0d69-4cf4-8dd3-6be4014b0e09/kube-auth-proxy/0.log"
Jul 30 23:39:21.236707 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:21.236676    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-n2d2v_b9801adf-19b8-4d27-b493-69577a9e4c33/payload-processing/0.log"
Jul 30 23:39:21.852499 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:21.852464    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6grqg_e086a0bb-93bd-4b2c-979b-c7c77628ae74/main/0.log"
Jul 30 23:39:21.973846 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:21.973809    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-p9wr2_33f08a97-68b6-49c0-b12e-59dcb6067222/main/0.log"
Jul 30 23:39:22.344710 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:22.344684    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-5lgzk_fec0f14c-a1a0-4b3c-95de-24ddb8efb612/main/0.log"
Jul 30 23:39:31.039636 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:31.039604    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-989b2_33ceed57-7068-43fe-82bd-ed7a69ce892d/global-pull-secret-syncer/0.log"
Jul 30 23:39:31.214081 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:31.214035    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wj9cn_32e5a3f3-ee4a-4723-ad23-2221405b4470/konnectivity-agent/0.log"
Jul 30 23:39:31.260909 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:31.260881    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-225.ec2.internal_1c4b33d732e720c94e138a741dc2f7ae/haproxy/0.log"
Jul 30 23:39:36.263260 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:36.263230    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-qpwsq_091be627-208c-484a-966b-f56076624fb4/manager/0.log"
Jul 30 23:39:36.293822 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:36.293793    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-6d7qw_c2deeb6e-cd26-4ffd-9eed-f830be5d6c77/manager/0.log"
Jul 30 23:39:36.318832 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:36.318808    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-xwqsq_80b354f0-4267-46c8-a78c-86f2fcb5425f/kuadrant-console-plugin/0.log"
Jul 30 23:39:36.406532 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:36.406497    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-68cf589589-q2hm7_06fd676b-515b-40a7-9205-a0623478c2ba/manager/0.log"
Jul 30 23:39:36.521107 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:36.521008    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-nfs8s_0c8e620f-9826-4761-9ab2-e99a790ce61e/manager/0.log"
Jul 30 23:39:38.384158 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:38.384085    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-48hk5_115e7dcd-3593-4794-8ebe-0022a1494f91/node-exporter/0.log"
Jul 30 23:39:38.404815 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:38.404787    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-48hk5_115e7dcd-3593-4794-8ebe-0022a1494f91/kube-rbac-proxy/0.log"
Jul 30 23:39:38.429788 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:38.429760    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-48hk5_115e7dcd-3593-4794-8ebe-0022a1494f91/init-textfile/0.log"
Jul 30 23:39:41.414181 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414141    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"]
Jul 30 23:39:41.414666 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414648    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6700b22b-3a64-42da-aa0a-86820218d15e" containerName="payload-pre-processing"
Jul 30 23:39:41.414741 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414668    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6700b22b-3a64-42da-aa0a-86820218d15e" containerName="payload-pre-processing"
Jul 30 23:39:41.414741 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414685    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6996cce5-35f4-4589-a6c5-7cf81ebab24d" containerName="revoke-keys"
Jul 30 23:39:41.414741 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414693    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6996cce5-35f4-4589-a6c5-7cf81ebab24d" containerName="revoke-keys"
Jul 30 23:39:41.414741 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414710    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14abd2ab-cd5d-4848-b181-e04c5a9c281e" containerName="payload-processing"
Jul 30 23:39:41.414741 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414719    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="14abd2ab-cd5d-4848-b181-e04c5a9c281e" containerName="payload-processing"
Jul 30 23:39:41.414988 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414815    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="14abd2ab-cd5d-4848-b181-e04c5a9c281e" containerName="payload-processing"
Jul 30 23:39:41.414988 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414830    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6700b22b-3a64-42da-aa0a-86820218d15e" containerName="payload-pre-processing"
Jul 30 23:39:41.414988 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.414839    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6996cce5-35f4-4589-a6c5-7cf81ebab24d" containerName="revoke-keys"
Jul 30 23:39:41.417828 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.417801    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.420529 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.420498    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nv5zt\"/\"kube-root-ca.crt\""
Jul 30 23:39:41.420680 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.420544    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nv5zt\"/\"openshift-service-ca.crt\""
Jul 30 23:39:41.421519 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.421503    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nv5zt\"/\"default-dockercfg-7rfhb\""
Jul 30 23:39:41.428882 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.428852    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"]
Jul 30 23:39:41.500407 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.500363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-podres\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.500600 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.500416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-sys\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.500600 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.500439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-lib-modules\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.500600 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.500508    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxjlg\" (UniqueName: \"kubernetes.io/projected/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-kube-api-access-gxjlg\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.500600 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.500555    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-proc\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.600968 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.600931    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-podres\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.601223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.600978    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-sys\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.601223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.600996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-lib-modules\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.601223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.601106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-sys\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.601223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.601129    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxjlg\" (UniqueName: \"kubernetes.io/projected/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-kube-api-access-gxjlg\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.601223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.601157    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-podres\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.601223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.601178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-proc\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.601223 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.601204    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-lib-modules\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.601599 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.601265    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-proc\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.608859 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.608836    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxjlg\" (UniqueName: \"kubernetes.io/projected/9a68db85-5c1c-40f3-99bb-3b80cddab5b6-kube-api-access-gxjlg\") pod \"perf-node-gather-daemonset-52z4v\" (UID: \"9a68db85-5c1c-40f3-99bb-3b80cddab5b6\") " pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.728225 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.728189    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:41.869819 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:41.869797    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"]
Jul 30 23:39:41.872627 ip-10-0-142-225 kubenswrapper[2578]: W0730 23:39:41.872603    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9a68db85_5c1c_40f3_99bb_3b80cddab5b6.slice/crio-065e60b1521358c47856b3c3ac3e35a88fd15cc845bdcdc0abc2914898aecb7f WatchSource:0}: Error finding container 065e60b1521358c47856b3c3ac3e35a88fd15cc845bdcdc0abc2914898aecb7f: Status 404 returned error can't find the container with id 065e60b1521358c47856b3c3ac3e35a88fd15cc845bdcdc0abc2914898aecb7f
Jul 30 23:39:42.738516 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:42.738473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v" event={"ID":"9a68db85-5c1c-40f3-99bb-3b80cddab5b6","Type":"ContainerStarted","Data":"8724615eae8349470e908a58266280edecb2859e713ca65aa8ec261fca8e1848"}
Jul 30 23:39:42.738902 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:42.738538    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:42.738902 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:42.738555    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v" event={"ID":"9a68db85-5c1c-40f3-99bb-3b80cddab5b6","Type":"ContainerStarted","Data":"065e60b1521358c47856b3c3ac3e35a88fd15cc845bdcdc0abc2914898aecb7f"}
Jul 30 23:39:42.754355 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:42.754299    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v" podStartSLOduration=1.754282637 podStartE2EDuration="1.754282637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-30 23:39:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-30 23:39:42.753323158 +0000 UTC m=+3562.928286844" watchObservedRunningTime="2026-07-30 23:39:42.754282637 +0000 UTC m=+3562.929246316"
Jul 30 23:39:43.485214 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:43.485184    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tm4q5_9007b6e3-863d-426c-b9c7-c2c3637861af/dns/0.log"
Jul 30 23:39:43.507146 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:43.507115    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tm4q5_9007b6e3-863d-426c-b9c7-c2c3637861af/kube-rbac-proxy/0.log"
Jul 30 23:39:43.577719 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:43.577683    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rn2w4_351a395f-318f-472a-918e-c463a6326d31/dns-node-resolver/0.log"
Jul 30 23:39:44.105890 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:44.105861    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6758c75797-xh58q_5cb49bc7-91fd-4326-ad9c-30dc316e9a98/registry/0.log"
Jul 30 23:39:44.127692 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:44.127670    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7xbdc_2cc5821f-7a57-4654-9f65-e309bd1d58b8/node-ca/0.log"
Jul 30 23:39:45.291910 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:45.291880    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b9c58d584-x4tth_f0ff893d-0d69-4cf4-8dd3-6be4014b0e09/kube-auth-proxy/0.log"
Jul 30 23:39:45.402026 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:45.401997    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-n2d2v_b9801adf-19b8-4d27-b493-69577a9e4c33/payload-processing/0.log"
Jul 30 23:39:46.210479 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:46.210437    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jzsrh_f931cf0f-d228-45b8-999a-171024393c99/serve-healthcheck-canary/0.log"
Jul 30 23:39:46.868009 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:46.867984    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rwrlr_c82b749b-fe1a-4132-81ec-042cad024f81/kube-rbac-proxy/0.log"
Jul 30 23:39:46.888945 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:46.888920    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rwrlr_c82b749b-fe1a-4132-81ec-042cad024f81/exporter/0.log"
Jul 30 23:39:46.913774 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:46.913750    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rwrlr_c82b749b-fe1a-4132-81ec-042cad024f81/extractor/0.log"
Jul 30 23:39:48.754531 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:48.754498    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nv5zt/perf-node-gather-daemonset-52z4v"
Jul 30 23:39:48.943810 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:48.943779    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wkz9w_7130dbaf-62b7-4d36-bbfb-a031aaa8f782/manager/0.log"
Jul 30 23:39:49.163120 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:49.163020    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jjs7h_1780e2f1-70d4-4b00-9a5f-595c19eaee9d/manager/0.log"
Jul 30 23:39:49.393737 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:49.393706    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5w7xx_b65d94e7-4b50-41ec-9c55-cfda5fc159c3/manager/0.log"
Jul 30 23:39:49.603166 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:49.603130    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff8c5f475-ghbrv_b671cb7b-cf47-4a48-b297-bd65c9b46755/manager/0.log"
Jul 30 23:39:49.625267 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:49.625240    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-ff8c5f475-ghbrv_b671cb7b-cf47-4a48-b297-bd65c9b46755/copy-manifests/0.log"
Jul 30 23:39:51.078425 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:51.078392    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-6lntc_e41dba6a-cad0-40de-af0c-e13a2ca8c041/openshift-lws-operator/0.log"
Jul 30 23:39:52.014415 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:52.014383    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b9c58d584-x4tth_f0ff893d-0d69-4cf4-8dd3-6be4014b0e09/kube-auth-proxy/0.log"
Jul 30 23:39:52.082321 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:52.082287    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-n2d2v_b9801adf-19b8-4d27-b493-69577a9e4c33/payload-processing/0.log"
Jul 30 23:39:53.742497 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:53.742466    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-b9c58d584-x4tth_f0ff893d-0d69-4cf4-8dd3-6be4014b0e09/kube-auth-proxy/0.log"
Jul 30 23:39:53.807622 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:53.807589    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-n2d2v_b9801adf-19b8-4d27-b493-69577a9e4c33/payload-processing/0.log"
Jul 30 23:39:57.565389 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:57.565311    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2tmpp_9a2e2f7c-5e17-4380-bf37-556b6b61eff0/kube-multus/0.log"
Jul 30 23:39:57.907252 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:57.907168    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq7vv_68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a/kube-multus-additional-cni-plugins/0.log"
Jul 30 23:39:57.929092 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:57.929063    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq7vv_68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a/egress-router-binary-copy/0.log"
Jul 30 23:39:57.950621 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:57.950592    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq7vv_68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a/cni-plugins/0.log"
Jul 30 23:39:57.975017 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:57.974989    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq7vv_68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a/bond-cni-plugin/0.log"
Jul 30 23:39:58.000802 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:58.000772    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq7vv_68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a/routeoverride-cni/0.log"
Jul 30 23:39:58.028970 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:58.028942    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq7vv_68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a/whereabouts-cni-bincopy/0.log"
Jul 30 23:39:58.054766 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:58.054741    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tq7vv_68f7ebb1-37ea-4bcc-9f3e-bc65f0c7704a/whereabouts-cni/0.log"
Jul 30 23:39:58.208194 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:58.208166    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-m7mw5_b062ccea-8395-4cf7-8ad8-6fe3011f432b/network-metrics-daemon/0.log"
Jul 30 23:39:58.231271 ip-10-0-142-225 kubenswrapper[2578]: I0730 23:39:58.231247    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-m7mw5_b062ccea-8395-4cf7-8ad8-6fe3011f432b/kube-rbac-proxy/0.log"