Jul 26 18:12:07.086113 ip-10-0-138-70 systemd[1]: Starting Kubernetes Kubelet...
Jul 26 18:12:08.898424 ip-10-0-138-70 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 18:12:08.898424 ip-10-0-138-70 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 26 18:12:08.898424 ip-10-0-138-70 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 18:12:08.898424 ip-10-0-138-70 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 26 18:12:08.898424 ip-10-0-138-70 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 26 18:12:08.899876 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.899782    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 26 18:12:08.903818 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903801    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:12:08.903818 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903817    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903822    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903825    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903828    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903831    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903834    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903837    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903840    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903842    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903846    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903848    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903851    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903853    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903856    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903858    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903861    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903863    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903866    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903868    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903873    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:12:08.903894 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903882    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903885    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903889    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903893    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903896    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903900    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903903    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903905    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903908    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903911    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903913    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903916    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903919    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903921    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903924    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903927    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903929    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903932    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903934    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903936    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:12:08.904366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903939    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903941    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903944    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903946    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903950    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903953    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903957    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903959    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903962    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903965    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903968    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903970    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903973    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903975    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903979    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903982    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903985    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903988    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903990    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903993    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:12:08.904908 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903996    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.903998    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904001    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904003    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904006    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904008    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904011    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904014    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904016    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904019    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904022    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904024    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904026    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904029    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904032    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904034    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904037    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904040    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904043    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904045    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:12:08.905384 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904048    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904050    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904053    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904055    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904058    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904061    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904065    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904069    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904071    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904074    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904076    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904079    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904081    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904084    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904087    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904090    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904092    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904095    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904823    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:12:08.906074 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904830    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904835    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904839    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904842    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904845    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904848    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904851    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904854    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904856    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904859    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904862    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904864    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904868    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904872    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904875    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904877    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904880    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904882    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904885    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:12:08.906827 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904888    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904892    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904894    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904897    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904900    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904903    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904905    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904908    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904910    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904913    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904916    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904918    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904921    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904923    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904926    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904929    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904932    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904934    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904937    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904940    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:12:08.907319 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904943    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904946    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904948    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904951    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904954    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904956    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904959    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904962    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904964    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904967    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904969    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904973    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904976    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904979    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904982    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904985    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904988    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904990    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904993    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904996    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:12:08.907847 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.904998    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905001    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905003    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905006    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905009    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905011    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905014    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905017    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905019    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905022    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905024    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905027    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905030    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905032    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905035    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905038    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905040    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905043    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905045    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905048    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:12:08.908327 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905050    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905053    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905055    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905058    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905061    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905064    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905066    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905069    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905072    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905074    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905077    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905079    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905082    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905084    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905087    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905089    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905092    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905096    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.905099    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906435    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:12:08.908859 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906448    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906453    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906457    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906462    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906467    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906472    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906482    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906486    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906491    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906495    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906499    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906504    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906508    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906513    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906517    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906521    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906527    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906531    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906541    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906547    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:12:08.909347 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906551    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906556    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906560    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906564    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906569    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906573    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906577    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906582    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906587    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906591    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906595    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906604    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906608    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906626    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906631    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906635    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906640    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906644    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906648    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:12:08.909862 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906653    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906657    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906662    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906666    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906671    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906680    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906684    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906689    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906694    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906698    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906702    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906707    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906711    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906717    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906721    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906725    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906737    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906743    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906747    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906752    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:12:08.910363 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906757    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906761    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906765    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906769    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906773    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906777    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906782    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906786    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906790    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906800    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906804    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906808    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906813    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906820    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906825    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906830    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906839    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906844    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906849    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906853    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:12:08.910871 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906863    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906868    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906872    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906876    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906880    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906885    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906890    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906894    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906898    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906905    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906910    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906914    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906920    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906923    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906927    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906930    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906933    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906936    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.906939    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907063    2559 flags.go:64] FLAG: --address="0.0.0.0"
Jul 26 18:12:08.911351 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907071    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907078    2559 flags.go:64] FLAG: --anonymous-auth="true"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907083    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907088    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907095    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907282    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907302    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907307    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907311    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907315    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907319    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907322    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907325    2559 flags.go:64] FLAG: --cgroup-root=""
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907329    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907333    2559 flags.go:64] FLAG: --client-ca-file=""
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907336    2559 flags.go:64] FLAG: --cloud-provider="external"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907339    2559 flags.go:64] FLAG: --cluster-dns="[]"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907345    2559 flags.go:64] FLAG: --cluster-domain=""
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907348    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907351    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907355    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907358    2559 flags.go:64] FLAG: --container-log-max-files="5"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907362    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 26 18:12:08.911933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907365    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907368    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907372    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907375    2559 flags.go:64] FLAG: --contention-profiling="false"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907378    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907381    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907385    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907388    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907393    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907396    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907399    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907402    2559 flags.go:64] FLAG: --enable-load-reader="false"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907405    2559 flags.go:64] FLAG: --enable-server="true"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907408    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907415    2559 flags.go:64] FLAG: --event-burst="100"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907418    2559 flags.go:64] FLAG: --event-qps="50"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907421    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907425    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907428    2559 flags.go:64] FLAG: --eviction-hard=""
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907432    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907435    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907438    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907442    2559 flags.go:64] FLAG: --eviction-soft=""
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907444    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907448    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 26 18:12:08.912503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907451    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907454    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907456    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907459    2559 flags.go:64] FLAG: --fail-swap-on="true"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907462    2559 flags.go:64] FLAG: --feature-gates=""
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907466    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907469    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907472    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907475    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907479    2559 flags.go:64] FLAG: --healthz-port="10248"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907482    2559 flags.go:64] FLAG: --help="false"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907484    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-138-70.ec2.internal"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907488    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907491    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907494    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907498    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907502    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907505    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907508    2559 flags.go:64] FLAG: --image-service-endpoint=""
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907511    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907514    2559 flags.go:64] FLAG: --kube-api-burst="100"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907518    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907522    2559 flags.go:64] FLAG: --kube-api-qps="50"
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907525    2559 flags.go:64] FLAG: --kube-reserved=""
Jul 26 18:12:08.913156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907528    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907531    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907534    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907537    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907540    2559 flags.go:64] FLAG: --lock-file=""
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907543    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907545    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907549    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907555    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907558    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907562    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907565    2559 flags.go:64] FLAG: --logging-format="text"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907568    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907571    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907574    2559 flags.go:64] FLAG: --manifest-url=""
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907577    2559 flags.go:64] FLAG: --manifest-url-header=""
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907582    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907585    2559 flags.go:64] FLAG: --max-open-files="1000000"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907590    2559 flags.go:64] FLAG: --max-pods="110"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907593    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907596    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907599    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907602    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907605    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907608    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 26 18:12:08.913787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907611    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907637    2559 flags.go:64] FLAG: --node-status-max-images="50"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907642    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907650    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907655    2559 flags.go:64] FLAG: --pod-cidr=""
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907658    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907665    2559 flags.go:64] FLAG: --pod-manifest-path=""
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907668    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907671    2559 flags.go:64] FLAG: --pods-per-core="0"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907674    2559 flags.go:64] FLAG: --port="10250"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907677    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907680    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-046a109925c883a1d"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907683    2559 flags.go:64] FLAG: --qos-reserved=""
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907686    2559 flags.go:64] FLAG: --read-only-port="10255"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907689    2559 flags.go:64] FLAG: --register-node="true"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907692    2559 flags.go:64] FLAG: --register-with-taints=""
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907696    2559 flags.go:64] FLAG: --registry-burst="10"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907699    2559 flags.go:64] FLAG: --registry-qps="5"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907702    2559 flags.go:64] FLAG: --reserved-cpus=""
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907705    2559 flags.go:64] FLAG: --reserved-memory=""
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907709    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907712    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907714    2559 flags.go:64] FLAG: --rotate-certificates="false"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907717    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907720    2559 flags.go:64] FLAG: --runonce="false"
Jul 26 18:12:08.914436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907723    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907726    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907729    2559 flags.go:64] FLAG: --seccomp-default="false"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907732    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907735    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907738    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907741    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907744    2559 flags.go:64] FLAG: --storage-driver-password="root"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907746    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907750    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907753    2559 flags.go:64] FLAG: --storage-driver-user="root"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907758    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907761    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907764    2559 flags.go:64] FLAG: --system-cgroups=""
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907767    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907774    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907777    2559 flags.go:64] FLAG: --tls-cert-file=""
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907779    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907784    2559 flags.go:64] FLAG: --tls-min-version=""
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907787    2559 flags.go:64] FLAG: --tls-private-key-file=""
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907790    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907793    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907796    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907799    2559 flags.go:64] FLAG: --v="2"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907803    2559 flags.go:64] FLAG: --version="false"
Jul 26 18:12:08.915058 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907807    2559 flags.go:64] FLAG: --vmodule=""
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907811    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.907815    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907938    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907945    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907949    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907952    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907955    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907958    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907960    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907963    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907965    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907968    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907971    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907974    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907977    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907980    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907982    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907988    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907991    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:12:08.915708 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907993    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907996    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.907999    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908001    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908004    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908007    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908009    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908012    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908015    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908017    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908020    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908023    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908026    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908028    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908031    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908035    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908039    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908042    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908044    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908047    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:12:08.916198 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908049    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908052    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908055    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908057    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908060    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908063    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908065    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908068    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908070    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908073    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908078    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908081    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908083    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908086    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908088    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908091    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908094    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908098    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908101    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908103    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:12:08.916695 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908106    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908108    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908111    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908113    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908116    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908119    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908121    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908125    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908127    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908130    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908133    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908135    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908138    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908140    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908142    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908145    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908147    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908150    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908153    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908155    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:12:08.917177 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908158    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908160    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908166    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908169    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908172    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908174    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908177    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908181    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908184    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908187    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908190    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908192    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908195    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908198    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908200    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908203    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908205    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908208    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908210    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:12:08.917678 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908213    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908216    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.908219    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.908827    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.915672    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.915693    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.916398    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.916406    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916467    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916473    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916476    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916480    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916483    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916486    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:12:08.918142 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916489    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916492    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916495    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916497    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916500    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916502    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916505    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916508    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916510    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916513    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916515    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916518    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916521    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916524    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916526    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916530    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916534    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916537    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916540    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916543    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:12:08.918508 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916546    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916549    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916552    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916557    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916560    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916563    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916565    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916569    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916571    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916574    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916576    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916579    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916582    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916584    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916587    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916590    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916592    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916595    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916597    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:12:08.919013 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916600    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916603    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916606    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916608    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916610    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916627    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916632    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916636    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916639    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916642    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916644    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916647    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916650    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916653    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916655    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916658    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916661    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916665    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916668    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916673    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:12:08.919473 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916677    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916679    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916682    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916685    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916688    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916690    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916693    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916695    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916698    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916701    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916703    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916706    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916709    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916711    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916714    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916718    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916722    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916724    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916727    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916730    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:12:08.920049 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916732    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916735    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916738    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916740    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916743    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916745    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916748    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916751    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916754    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916757    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916760    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916763    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916765    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916768    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.916773    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 18:12:08.920535 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916892    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916898    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916901    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916904    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916907    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916909    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916912    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916915    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916918    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916920    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916923    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916925    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916928    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916931    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916933    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916936    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916938    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916941    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916943    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916946    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 26 18:12:08.920932 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916948    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916951    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916953    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916956    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916959    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916962    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916965    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916967    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916971    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916973    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916976    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916979    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916981    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916985    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916989    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916992    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916994    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.916998    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917002    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 26 18:12:08.921414 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917005    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917008    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917011    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917013    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917016    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917018    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917021    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917024    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917026    2559 feature_gate.go:328] unrecognized feature gate: Example
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917029    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917031    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917034    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917037    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917040    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917043    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917045    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917048    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917051    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917053    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917057    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 26 18:12:08.921971 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917059    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917062    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917065    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917068    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917070    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917073    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917076    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917078    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917081    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917083    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917086    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917088    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917091    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917093    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917096    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917098    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917101    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917104    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917106    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917109    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 26 18:12:08.922459 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917111    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917114    2559 feature_gate.go:328] unrecognized feature gate: Example2
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917117    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917120    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917124    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917127    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917130    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917132    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917135    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917138    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917140    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917143    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917146    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917149    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917151    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917155    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917157    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917160    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917162    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 26 18:12:08.922958 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:08.917165    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 26 18:12:08.923434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.917170    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 26 18:12:08.923434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.917318    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 26 18:12:08.923434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.919639    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 26 18:12:08.923434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.920863    2559 server.go:1019] "Starting client certificate rotation"
Jul 26 18:12:08.923434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.920968    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 18:12:08.923434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.921644    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 26 18:12:08.923434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.923071    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 18:12:08.924932 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.924909    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 26 18:12:08.934259 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.934235    2559 log.go:25] "Validated CRI v1 runtime API"
Jul 26 18:12:08.937855 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.937835    2559 log.go:25] "Validated CRI v1 image API"
Jul 26 18:12:08.939780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.939764    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 26 18:12:08.942421 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.942394    2559 fs.go:135] Filesystem UUIDs: map[134749b9-e2f1-4c46-a425-fc9ab02f9312:/dev/nvme0n1p3 609f4f60-db34-48e4-8a27-133bd5802cc8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 26 18:12:08.942473 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.942421    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 26 18:12:08.948183 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.948061    2559 manager.go:216] Machine: {Timestamp:2026-07-26 18:12:08.946736213 +0000 UTC m=+0.271258301 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098811 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b3a112b26a4fa2f94d64456e007dc SystemUUID:ec2b3a11-2b26-a4fa-2f94-d64456e007dc BootID:9dfc66b0-9147-4a19-a8a0-5997e83bab5e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7f:6a:14:36:8b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7f:6a:14:36:8b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:ba:91:26:4a:fe Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 26 18:12:08.948183 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.948181    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 26 18:12:08.948287 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.948269    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 26 18:12:08.951222 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.951193    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 26 18:12:08.951375 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.951227    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-70.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 26 18:12:08.951893 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.951882    2559 topology_manager.go:138] "Creating topology manager with none policy"
Jul 26 18:12:08.951932 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.951896    2559 container_manager_linux.go:309] "Creating device plugin manager"
Jul 26 18:12:08.951932 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.951905    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 18:12:08.951932 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.951918    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 26 18:12:08.953054 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.953036    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 18:12:08.953592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.953580    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 26 18:12:08.957369 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.957356    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 18:12:08.957497 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.957487    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 26 18:12:08.957544 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.957518    2559 kubelet.go:486] "Attempting to sync node with API server"
Jul 26 18:12:08.957544 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.957529    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 26 18:12:08.957544 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.957544    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 26 18:12:08.957709 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.957554    2559 kubelet.go:398] "Adding apiserver pod source"
Jul 26 18:12:08.957709 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.957564    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 26 18:12:08.958079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.958065    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 18:12:08.958134 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.958089    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 26 18:12:08.960261 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.960239    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 26 18:12:08.961604 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.961590    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 26 18:12:08.961680 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.961637    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 26 18:12:08.962271 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962259    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 26 18:12:08.962329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962275    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 26 18:12:08.962329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962281    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 26 18:12:08.962329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962287    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 26 18:12:08.962329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962296    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 26 18:12:08.962329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962301    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 26 18:12:08.962329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962309    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 26 18:12:08.962329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962314    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 26 18:12:08.962329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962322    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 26 18:12:08.962329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962327    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 26 18:12:08.962556 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962335    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 26 18:12:08.962556 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962340    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 26 18:12:08.962945 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962936    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 26 18:12:08.962945 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.962944    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 26 18:12:08.966278 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.966265    2559 server.go:1268] "Started kubelet"
Jul 26 18:12:08.966376 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.966353    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 26 18:12:08.966460 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.966417    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 26 18:12:08.966520 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.966481    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 26 18:12:08.968316 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.968056    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 26 18:12:08.968437 ip-10-0-138-70 systemd[1]: Started Kubernetes Kubelet.
Jul 26 18:12:08.968557 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.968439    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-70.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 26 18:12:08.968557 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:08.968437    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-70.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 26 18:12:08.968557 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:08.968534    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 26 18:12:08.970346 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.970327    2559 server.go:310] "Adding debug handlers to kubelet server"
Jul 26 18:12:08.971329 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:08.971304    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 26 18:12:08.974610 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:08.973846    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-70.ec2.internal.18c5e978024d510c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-70.ec2.internal,UID:ip-10-0-138-70.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-70.ec2.internal,},FirstTimestamp:2026-07-26 18:12:08.966238476 +0000 UTC m=+0.290760563,LastTimestamp:2026-07-26 18:12:08.966238476 +0000 UTC m=+0.290760563,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-70.ec2.internal,}"
Jul 26 18:12:08.978279 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.978259    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qvjc6"
Jul 26 18:12:08.978780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.978756    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 26 18:12:08.978865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.978777    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 26 18:12:08.979038 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:08.978937    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:08.979311 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.979295    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 26 18:12:08.979396 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.979350    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 26 18:12:08.979396 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.979365    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 26 18:12:08.979532 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.979513    2559 reconstruct.go:97] "Volume reconstruction finished"
Jul 26 18:12:08.979532 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.979527    2559 reconciler.go:29] "Reconciler: start to sync state"
Jul 26 18:12:08.980226 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.980210    2559 factory.go:55] Registering systemd factory
Jul 26 18:12:08.980226 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.980227    2559 factory.go:223] Registration of the systemd container factory successfully
Jul 26 18:12:08.980462 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.980446    2559 factory.go:153] Registering CRI-O factory
Jul 26 18:12:08.980462 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.980462    2559 factory.go:223] Registration of the crio container factory successfully
Jul 26 18:12:08.980573 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.980517    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 26 18:12:08.980573 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.980536    2559 factory.go:103] Registering Raw factory
Jul 26 18:12:08.980573 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.980547    2559 manager.go:1196] Started watching for new ooms in manager
Jul 26 18:12:08.981161 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:08.981121    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-70.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 26 18:12:08.981281 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:08.981259    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 26 18:12:08.981366 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.981297    2559 manager.go:349] Starting recovery of all containers
Jul 26 18:12:08.986194 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.986158    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qvjc6"
Jul 26 18:12:08.990070 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.990052    2559 manager.go:354] Recovery completed
Jul 26 18:12:08.994269 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.994253    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:12:08.996311 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.996295    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:12:08.996390 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.996327    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:12:08.996390 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.996342    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:12:08.996981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.996965    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 26 18:12:08.996981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.996981    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 26 18:12:08.997093 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.996998    2559 state_mem.go:36] "Initialized new in-memory state store"
Jul 26 18:12:08.997093 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.996969    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:12:08.997816 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.997802    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:12:08.997879 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.997828    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:12:08.997879 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.997838    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:12:08.998937 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.998925    2559 policy_none.go:49] "None policy: Start"
Jul 26 18:12:08.998994 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.998943    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 26 18:12:08.998994 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:08.998953    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 26 18:12:09.000848 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.000836    2559 policy_none.go:47] "Start"
Jul 26 18:12:09.040440 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.040225    2559 manager.go:337] "Starting Device Plugin manager"
Jul 26 18:12:09.040584 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.040468    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 26 18:12:09.040584 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.040481    2559 server.go:87] "Starting device plugin registration server"
Jul 26 18:12:09.040786 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.040774    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 26 18:12:09.040820 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.040789    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 26 18:12:09.040989 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.040920    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 26 18:12:09.041100 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.041006    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 26 18:12:09.041100 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.041014    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 26 18:12:09.041531 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.041510    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 26 18:12:09.041643 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.041558    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.141943 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.141844    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:12:09.143923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.142958    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:12:09.143923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.142990    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:12:09.143923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.143002    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:12:09.143923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.143030    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.149019 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.149003    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.149071 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.149028    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-70.ec2.internal\": node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.163240 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.163213    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.168465 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.168440    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 26 18:12:09.169749 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.169726    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 26 18:12:09.169849 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.169766    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 26 18:12:09.169849 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.169788    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 26 18:12:09.169849 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.169829    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 26 18:12:09.172717 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.172699    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:12:09.263967 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.263911    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.270085 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.270057    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"]
Jul 26 18:12:09.270146 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.270138    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:12:09.271213 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.271195    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:12:09.271316 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.271227    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:12:09.271316 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.271237    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:12:09.273399 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.273386    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:12:09.273575 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.273560    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.273631 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.273592    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:12:09.274676 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.274661    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:12:09.274740 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.274691    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:12:09.274740 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.274706    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:12:09.275310 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.275295    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:12:09.275401 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.275323    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:12:09.275401 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.275338    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:12:09.277404 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.277388    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.277511 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.277416    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 26 18:12:09.278129 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.278113    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientMemory"
Jul 26 18:12:09.278206 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.278149    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasNoDiskPressure"
Jul 26 18:12:09.278206 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.278176    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeHasSufficientPID"
Jul 26 18:12:09.281069 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.281052    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0771439e9202664f9575f6e0ac07a38-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal\" (UID: \"a0771439e9202664f9575f6e0ac07a38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.281130 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.281077    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a2a497afdef47058f3109e35eaa2548a-config\") pod \"kube-apiserver-proxy-ip-10-0-138-70.ec2.internal\" (UID: \"a2a497afdef47058f3109e35eaa2548a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.281130 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.281093    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a0771439e9202664f9575f6e0ac07a38-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal\" (UID: \"a0771439e9202664f9575f6e0ac07a38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.298497 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.298475    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-70.ec2.internal\" not found" node="ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.303049 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.303031    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-70.ec2.internal\" not found" node="ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.364513 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.364476    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.381421 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.381390    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0771439e9202664f9575f6e0ac07a38-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal\" (UID: \"a0771439e9202664f9575f6e0ac07a38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.381535 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.381431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a2a497afdef47058f3109e35eaa2548a-config\") pod \"kube-apiserver-proxy-ip-10-0-138-70.ec2.internal\" (UID: \"a2a497afdef47058f3109e35eaa2548a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.381535 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.381454    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a0771439e9202664f9575f6e0ac07a38-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal\" (UID: \"a0771439e9202664f9575f6e0ac07a38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.381535 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.381503    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0771439e9202664f9575f6e0ac07a38-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal\" (UID: \"a0771439e9202664f9575f6e0ac07a38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.381535 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.381514    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a2a497afdef47058f3109e35eaa2548a-config\") pod \"kube-apiserver-proxy-ip-10-0-138-70.ec2.internal\" (UID: \"a2a497afdef47058f3109e35eaa2548a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.381691 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.381545    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a0771439e9202664f9575f6e0ac07a38-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal\" (UID: \"a0771439e9202664f9575f6e0ac07a38\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.464812 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.464778    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.565305 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.565264    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.600808 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.600779    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.606229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.605953    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:09.665404 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.665365    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.765882 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.765802    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.815906 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.815874    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:12:09.866272 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.866238    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.920804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.920757    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 26 18:12:09.921368 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.920980    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 18:12:09.921368 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.920983    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 26 18:12:09.967388 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:09.967345    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:09.991493 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.991455    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-25 18:07:08 +0000 UTC" deadline="2028-02-13 01:30:19.111509446 +0000 UTC"
Jul 26 18:12:09.991493 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:09.991490    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13591h18m9.120024708s"
Jul 26 18:12:10.067934 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:10.067902    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:10.131212 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:10.131176    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0771439e9202664f9575f6e0ac07a38.slice/crio-58e969956382e09123ef925aa8c1b504858072964173bff96a94d86591cb0b32 WatchSource:0}: Error finding container 58e969956382e09123ef925aa8c1b504858072964173bff96a94d86591cb0b32: Status 404 returned error can't find the container with id 58e969956382e09123ef925aa8c1b504858072964173bff96a94d86591cb0b32
Jul 26 18:12:10.136819 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.136791    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:12:10.142266 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:10.142242    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a497afdef47058f3109e35eaa2548a.slice/crio-f1e3ae3b0ba98ea8eac9dbadfcad2d82c8c4d37b00d179ecfbe39eac3ed9d20e WatchSource:0}: Error finding container f1e3ae3b0ba98ea8eac9dbadfcad2d82c8c4d37b00d179ecfbe39eac3ed9d20e: Status 404 returned error can't find the container with id f1e3ae3b0ba98ea8eac9dbadfcad2d82c8c4d37b00d179ecfbe39eac3ed9d20e
Jul 26 18:12:10.168096 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:10.168055    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:10.173709 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.173661    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal" event={"ID":"a0771439e9202664f9575f6e0ac07a38","Type":"ContainerStarted","Data":"58e969956382e09123ef925aa8c1b504858072964173bff96a94d86591cb0b32"}
Jul 26 18:12:10.174795 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.174772    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal" event={"ID":"a2a497afdef47058f3109e35eaa2548a","Type":"ContainerStarted","Data":"f1e3ae3b0ba98ea8eac9dbadfcad2d82c8c4d37b00d179ecfbe39eac3ed9d20e"}
Jul 26 18:12:10.268242 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:10.268202    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-70.ec2.internal\" not found"
Jul 26 18:12:10.287559 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.287489    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:12:10.330546 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.330522    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:12:10.378995 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.378962    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:10.389465 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.389441    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 18:12:10.390505 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.390490    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal"
Jul 26 18:12:10.400869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.400848    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 26 18:12:10.958332 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.958303    2559 apiserver.go:52] "Watching apiserver"
Jul 26 18:12:10.970463 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.970428    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 26 18:12:10.971492 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.971461    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-hm6m9","kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qmc5z","openshift-image-registry/node-ca-5bt9b","openshift-network-operator/iptables-alerter-8j8xj","openshift-ovn-kubernetes/ovnkube-node-7pqql","kube-system/global-pull-secret-syncer-n4ng2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw","openshift-dns/node-resolver-54nt6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal","openshift-multus/multus-4x2kk","openshift-multus/multus-additional-cni-plugins-xq4cx","openshift-multus/network-metrics-daemon-jlkb4","openshift-network-diagnostics/network-check-target-pmgnx"]
Jul 26 18:12:10.974147 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.974069    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:10.974302 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:10.974242    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:10.976365 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.976322    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:10.976484 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:10.976396    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:10.979033 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.978565    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.979764 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.979728    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 26 18:12:10.981479 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.981456    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 26 18:12:10.981806 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.981664    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 26 18:12:10.981806 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.981729    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-scfts\""
Jul 26 18:12:10.981959 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.981850    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:10.984346 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.984311    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 26 18:12:10.984486 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.984444    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:10.984557 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.984503    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m5fkb\""
Jul 26 18:12:10.984633 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.984555    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 26 18:12:10.984825 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.984808    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 26 18:12:10.986356 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.986337    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:10.987278 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.987250    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 26 18:12:10.987383 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.987330    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 26 18:12:10.987477 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.987464    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gp8gb\""
Jul 26 18:12:10.987866 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.987847    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 26 18:12:10.989299 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989158    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-run\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.989299 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989192    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-sys\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.989299 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989236    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-modprobe-d\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.989299 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989290    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-sysconfig\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.989562 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989329    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-var-lib-kubelet\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.989562 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989355    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-host\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.989562 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989393    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfr95\" (UniqueName: \"kubernetes.io/projected/74fb6e83-4639-444c-8414-f7f7d07d9ddb-kube-api-access-tfr95\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.989562 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989423    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35a8bada-5aff-496e-ad10-c359a3e6e416-serviceca\") pod \"node-ca-5bt9b\" (UID: \"35a8bada-5aff-496e-ad10-c359a3e6e416\") " pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:10.989562 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989472    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-kubelet-config\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:10.989562 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989550    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:10.989869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:10.989925 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989875    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-kubernetes\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.989925 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989897    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35a8bada-5aff-496e-ad10-c359a3e6e416-host\") pod \"node-ca-5bt9b\" (UID: \"35a8bada-5aff-496e-ad10-c359a3e6e416\") " pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:10.989925 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989912    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfn9b\" (UniqueName: \"kubernetes.io/projected/35a8bada-5aff-496e-ad10-c359a3e6e416-kube-api-access-gfn9b\") pod \"node-ca-5bt9b\" (UID: \"35a8bada-5aff-496e-ad10-c359a3e6e416\") " pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:10.990066 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989925    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-sysctl-d\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.990066 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-lib-modules\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.990066 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.989994    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-tuned\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.990066 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990018    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74fb6e83-4639-444c-8414-f7f7d07d9ddb-tmp\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.990066 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-dbus\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:10.990381 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990082    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:10.990381 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7sth\" (UniqueName: \"kubernetes.io/projected/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-kube-api-access-r7sth\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:10.990381 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990143    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-sysctl-conf\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.990381 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990178    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-systemd\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:10.991067 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990428    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-t6z9t\""
Jul 26 18:12:10.991067 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990472    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 26 18:12:10.991067 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990484    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 26 18:12:10.991067 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990437    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 26 18:12:10.991067 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990864    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 26 18:12:10.991067 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990896    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 26 18:12:10.991067 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.990936    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 26 18:12:10.991963 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.991704    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:10.992326 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.992307    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7xpsm\""
Jul 26 18:12:10.992427 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.992338    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 26 18:12:10.992427 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.992341    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 26 18:12:10.993581 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.993535    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 26 18:12:10.994019 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.994001    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:10.994101 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:10.994072    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:10.994168 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.994104    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 26 18:12:10.995032 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.994966    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hxm7s\""
Jul 26 18:12:10.995032 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.995024    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 26 18:12:10.996695 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.996534    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:10.999542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.999091    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:10.999542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.999198    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 26 18:12:10.999542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.999203    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 26 18:12:10.999764 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.999679    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q97xj\""
Jul 26 18:12:10.999764 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:10.999712    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 26 18:12:11.001675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.001653    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.002339 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.002200    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 26 18:12:11.002339 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.002212    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 26 18:12:11.002485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.002439    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 26 18:12:11.002972 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.002781    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sbqfq\""
Jul 26 18:12:11.002972 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.002966    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 26 18:12:11.004096 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.004075    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cgjb9\""
Jul 26 18:12:11.004188 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.004098    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 26 18:12:11.004255 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.004185    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 26 18:12:11.012857 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.012833    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cv9h8"
Jul 26 18:12:11.022298 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.022268    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cv9h8"
Jul 26 18:12:11.080439 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.080408    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 26 18:12:11.091005 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.090969    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/943739a7-45a0-444f-86dd-f0f5722ee83d-hosts-file\") pod \"node-resolver-54nt6\" (UID: \"943739a7-45a0-444f-86dd-f0f5722ee83d\") " pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.091005 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-cni-netd\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.091204 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091030    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d640dee1-af4a-450f-8d66-924e7d08dd98-env-overrides\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.091204 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091071    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d640dee1-af4a-450f-8d66-924e7d08dd98-ovn-node-metrics-cert\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.091204 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-modprobe-d\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.091204 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091169    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skqgc\" (UniqueName: \"kubernetes.io/projected/3a23d815-2cef-43ba-91f1-56e9fe83a282-kube-api-access-skqgc\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.091358 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091215    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb8e778a-bb91-4e9a-b7bb-d948def35021-cni-binary-copy\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.091358 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35a8bada-5aff-496e-ad10-c359a3e6e416-host\") pod \"node-ca-5bt9b\" (UID: \"35a8bada-5aff-496e-ad10-c359a3e6e416\") " pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:11.091358 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091242    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-modprobe-d\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.091358 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091281    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/978ce3f4-995f-41dc-a234-d77304a2a87d-host-slash\") pod \"iptables-alerter-8j8xj\" (UID: \"978ce3f4-995f-41dc-a234-d77304a2a87d\") " pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.091358 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091289    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/35a8bada-5aff-496e-ad10-c359a3e6e416-host\") pod \"node-ca-5bt9b\" (UID: \"35a8bada-5aff-496e-ad10-c359a3e6e416\") " pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:11.091358 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091312    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-slash\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.091358 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091327    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-node-log\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.091358 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-log-socket\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-var-lib-cni-multus\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091385    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-system-cni-dir\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091430    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-os-release\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091468    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-lib-modules\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091488    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/943739a7-45a0-444f-86dd-f0f5722ee83d-tmp-dir\") pod \"node-resolver-54nt6\" (UID: \"943739a7-45a0-444f-86dd-f0f5722ee83d\") " pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091504    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/978ce3f4-995f-41dc-a234-d77304a2a87d-iptables-alerter-script\") pod \"iptables-alerter-8j8xj\" (UID: \"978ce3f4-995f-41dc-a234-d77304a2a87d\") " pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091519    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-cni-bin\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091539    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-socket-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091561    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-cni-dir\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091576    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb8e778a-bb91-4e9a-b7bb-d948def35021-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-lib-modules\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091592    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcn69\" (UniqueName: \"kubernetes.io/projected/fb8e778a-bb91-4e9a-b7bb-d948def35021-kube-api-access-xcn69\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091643    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d640dee1-af4a-450f-8d66-924e7d08dd98-ovnkube-script-lib\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-dbus\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091694    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-systemd\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.091762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091722    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35a8bada-5aff-496e-ad10-c359a3e6e416-serviceca\") pod \"node-ca-5bt9b\" (UID: \"35a8bada-5aff-496e-ad10-c359a3e6e416\") " pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091791    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-run-netns\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091805    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-systemd\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091825    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-device-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091868    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-run-netns\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091893    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d640dee1-af4a-450f-8d66-924e7d08dd98-ovnkube-config\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091927    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-etc-selinux\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091903    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-dbus\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091966    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-run-k8s-cni-cncf-io\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.091995    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-daemon-config\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092019    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npkwz\" (UniqueName: \"kubernetes.io/projected/978ce3f4-995f-41dc-a234-d77304a2a87d-kube-api-access-npkwz\") pod \"iptables-alerter-8j8xj\" (UID: \"978ce3f4-995f-41dc-a234-d77304a2a87d\") " pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092040    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-kubelet\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092064    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9grbh\" (UniqueName: \"kubernetes.io/projected/d640dee1-af4a-450f-8d66-924e7d08dd98-kube-api-access-9grbh\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-var-lib-cni-bin\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092137    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh888\" (UniqueName: \"kubernetes.io/projected/943739a7-45a0-444f-86dd-f0f5722ee83d-kube-api-access-sh888\") pod \"node-resolver-54nt6\" (UID: \"943739a7-45a0-444f-86dd-f0f5722ee83d\") " pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.092304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092158    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/35a8bada-5aff-496e-ad10-c359a3e6e416-serviceca\") pod \"node-ca-5bt9b\" (UID: \"35a8bada-5aff-496e-ad10-c359a3e6e416\") " pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-systemd-units\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092188    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-var-lib-kubelet\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-hostroot\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5649c\" (UniqueName: \"kubernetes.io/projected/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-kube-api-access-5649c\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092271    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-var-lib-kubelet\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092297    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-run-systemd\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092325    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-sysconfig\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092351    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-run-ovn\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092375    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-run-ovn-kubernetes\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092354    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-var-lib-kubelet\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092399    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-socket-dir-parent\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092434    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-sysconfig\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092436    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-kubelet-config\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092471    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092497    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-kubelet-config\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092497    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-kubernetes\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092521    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfn9b\" (UniqueName: \"kubernetes.io/projected/35a8bada-5aff-496e-ad10-c359a3e6e416-kube-api-access-gfn9b\") pod \"node-ca-5bt9b\" (UID: \"35a8bada-5aff-496e-ad10-c359a3e6e416\") " pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092543    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e7807e6d-3df2-4084-a6ea-a05859dd6eda-konnectivity-ca\") pod \"konnectivity-agent-hm6m9\" (UID: \"e7807e6d-3df2-4084-a6ea-a05859dd6eda\") " pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-cnibin\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092575    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-tuned\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092593    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb8e778a-bb91-4e9a-b7bb-d948def35021-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092543    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-kubernetes\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.092601    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092635    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sth\" (UniqueName: \"kubernetes.io/projected/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-kube-api-access-r7sth\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092663    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-sysctl-conf\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-run\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.092725    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret podName:b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:11.592683399 +0000 UTC m=+2.917205495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret") pod "global-pull-secret-syncer-n4ng2" (UID: "b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092728    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-run\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092751    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-sys\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-host\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092792    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-system-cni-dir\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092838    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfr95\" (UniqueName: \"kubernetes.io/projected/74fb6e83-4639-444c-8414-f7f7d07d9ddb-kube-api-access-tfr95\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.093550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092873    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e7807e6d-3df2-4084-a6ea-a05859dd6eda-agent-certs\") pod \"konnectivity-agent-hm6m9\" (UID: \"e7807e6d-3df2-4084-a6ea-a05859dd6eda\") " pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092854    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-sysctl-conf\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092897    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-host\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092900    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-var-lib-openvswitch\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092924    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092948    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-sys\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092947    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-etc-openvswitch\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092978    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-run-openvswitch\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.092998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-os-release\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093017    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-etc-kubernetes\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093052    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-cnibin\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093083    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-registration-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093122    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-sys-fs\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093172    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-run-multus-certs\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093203    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-sysctl-d\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093228    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74fb6e83-4639-444c-8414-f7f7d07d9ddb-tmp\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093252    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.094079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093279    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-cni-binary-copy\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.094740 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093302    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-conf-dir\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.094740 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093316    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-sysctl-d\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.094740 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093331    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.094740 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.093359    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:11.094740 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.093498    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:11.094740 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.093556    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs podName:618d2dc6-77cc-49e1-a892-b3f1f0a4eea4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:11.593542897 +0000 UTC m=+2.918064972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs") pod "network-metrics-daemon-jlkb4" (UID: "618d2dc6-77cc-49e1-a892-b3f1f0a4eea4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:11.095917 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.095895    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74fb6e83-4639-444c-8414-f7f7d07d9ddb-tmp\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.096014 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.095923    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/74fb6e83-4639-444c-8414-f7f7d07d9ddb-etc-tuned\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.104675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.104640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfr95\" (UniqueName: \"kubernetes.io/projected/74fb6e83-4639-444c-8414-f7f7d07d9ddb-kube-api-access-tfr95\") pod \"tuned-qmc5z\" (UID: \"74fb6e83-4639-444c-8414-f7f7d07d9ddb\") " pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.104806 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.104684    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfn9b\" (UniqueName: \"kubernetes.io/projected/35a8bada-5aff-496e-ad10-c359a3e6e416-kube-api-access-gfn9b\") pod \"node-ca-5bt9b\" (UID: \"35a8bada-5aff-496e-ad10-c359a3e6e416\") " pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:11.104981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.104964    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7sth\" (UniqueName: \"kubernetes.io/projected/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-kube-api-access-r7sth\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:11.194303 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-var-lib-kubelet\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194490 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194317    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-hostroot\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194490 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194339    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-var-lib-kubelet\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194490 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194346    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5649c\" (UniqueName: \"kubernetes.io/projected/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-kube-api-access-5649c\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194490 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-run-systemd\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194490 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194413    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-run-ovn\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194490 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194429    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-hostroot\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194490 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194438    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-run-ovn-kubernetes\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194490 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194462    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-socket-dir-parent\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194490 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194481    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-run-ovn\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194506    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e7807e6d-3df2-4084-a6ea-a05859dd6eda-konnectivity-ca\") pod \"konnectivity-agent-hm6m9\" (UID: \"e7807e6d-3df2-4084-a6ea-a05859dd6eda\") " pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194520    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-run-ovn-kubernetes\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194530    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-cnibin\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194583    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-cnibin\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194540    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-socket-dir-parent\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194590    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb8e778a-bb91-4e9a-b7bb-d948def35021-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194508    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-run-systemd\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194672    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-system-cni-dir\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194694    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e7807e6d-3df2-4084-a6ea-a05859dd6eda-agent-certs\") pod \"konnectivity-agent-hm6m9\" (UID: \"e7807e6d-3df2-4084-a6ea-a05859dd6eda\") " pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-var-lib-openvswitch\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194739    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-etc-openvswitch\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194759    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-system-cni-dir\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194762    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-run-openvswitch\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194798    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-run-openvswitch\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194800    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-os-release\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194830    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-etc-kubernetes\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.194915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194853    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-cnibin\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194856    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-os-release\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194879    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-registration-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194903    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-sys-fs\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-run-multus-certs\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194962    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.194984    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-cni-binary-copy\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195007    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-conf-dir\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195075    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/943739a7-45a0-444f-86dd-f0f5722ee83d-hosts-file\") pod \"node-resolver-54nt6\" (UID: \"943739a7-45a0-444f-86dd-f0f5722ee83d\") " pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195090    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-var-lib-openvswitch\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195098    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-cni-netd\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195122    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-etc-openvswitch\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195122    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d640dee1-af4a-450f-8d66-924e7d08dd98-env-overrides\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d640dee1-af4a-450f-8d66-924e7d08dd98-ovn-node-metrics-cert\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195168    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-cnibin\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195177    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skqgc\" (UniqueName: \"kubernetes.io/projected/3a23d815-2cef-43ba-91f1-56e9fe83a282-kube-api-access-skqgc\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.195775 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195181    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e7807e6d-3df2-4084-a6ea-a05859dd6eda-konnectivity-ca\") pod \"konnectivity-agent-hm6m9\" (UID: \"e7807e6d-3df2-4084-a6ea-a05859dd6eda\") " pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195194    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-sys-fs\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195203    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-conf-dir\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195204    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/943739a7-45a0-444f-86dd-f0f5722ee83d-hosts-file\") pod \"node-resolver-54nt6\" (UID: \"943739a7-45a0-444f-86dd-f0f5722ee83d\") " pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195186    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fb8e778a-bb91-4e9a-b7bb-d948def35021-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb8e778a-bb91-4e9a-b7bb-d948def35021-cni-binary-copy\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195248    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-etc-kubernetes\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195250    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-kubelet-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-registration-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195295    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/978ce3f4-995f-41dc-a234-d77304a2a87d-host-slash\") pod \"iptables-alerter-8j8xj\" (UID: \"978ce3f4-995f-41dc-a234-d77304a2a87d\") " pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195326    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-slash\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195342    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-cni-netd\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195353    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-node-log\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195387    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-node-log\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195423    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/978ce3f4-995f-41dc-a234-d77304a2a87d-host-slash\") pod \"iptables-alerter-8j8xj\" (UID: \"978ce3f4-995f-41dc-a234-d77304a2a87d\") " pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195500    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195538    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-slash\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.196750 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195582    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-log-socket\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195655    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-var-lib-cni-multus\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-log-socket\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195688    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-run-multus-certs\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195733    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-system-cni-dir\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195773    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-system-cni-dir\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195814    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-os-release\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195817    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-cni-binary-copy\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195852    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/943739a7-45a0-444f-86dd-f0f5722ee83d-tmp-dir\") pod \"node-resolver-54nt6\" (UID: \"943739a7-45a0-444f-86dd-f0f5722ee83d\") " pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195852    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb8e778a-bb91-4e9a-b7bb-d948def35021-cni-binary-copy\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195858    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-var-lib-cni-multus\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195869    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fb8e778a-bb91-4e9a-b7bb-d948def35021-os-release\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195896    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/978ce3f4-995f-41dc-a234-d77304a2a87d-iptables-alerter-script\") pod \"iptables-alerter-8j8xj\" (UID: \"978ce3f4-995f-41dc-a234-d77304a2a87d\") " pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195960    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-cni-bin\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195978    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d640dee1-af4a-450f-8d66-924e7d08dd98-env-overrides\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.195981    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-socket-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196019    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-cni-dir\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.197452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196021    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-cni-bin\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196052    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-socket-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196079    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-cni-dir\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196121    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb8e778a-bb91-4e9a-b7bb-d948def35021-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196152    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/943739a7-45a0-444f-86dd-f0f5722ee83d-tmp-dir\") pod \"node-resolver-54nt6\" (UID: \"943739a7-45a0-444f-86dd-f0f5722ee83d\") " pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcn69\" (UniqueName: \"kubernetes.io/projected/fb8e778a-bb91-4e9a-b7bb-d948def35021-kube-api-access-xcn69\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196239    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d640dee1-af4a-450f-8d66-924e7d08dd98-ovnkube-script-lib\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196357    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-run-netns\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196398    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196425    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-device-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196459    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-run-netns\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d640dee1-af4a-450f-8d66-924e7d08dd98-ovnkube-config\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196511    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-etc-selinux\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196537    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/978ce3f4-995f-41dc-a234-d77304a2a87d-iptables-alerter-script\") pod \"iptables-alerter-8j8xj\" (UID: \"978ce3f4-995f-41dc-a234-d77304a2a87d\") " pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196566    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196632    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-run-k8s-cni-cncf-io\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196650    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fb8e778a-bb91-4e9a-b7bb-d948def35021-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.198180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196661    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-daemon-config\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196666    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-run-k8s-cni-cncf-io\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196711    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-device-dir\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npkwz\" (UniqueName: \"kubernetes.io/projected/978ce3f4-995f-41dc-a234-d77304a2a87d-kube-api-access-npkwz\") pod \"iptables-alerter-8j8xj\" (UID: \"978ce3f4-995f-41dc-a234-d77304a2a87d\") " pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196756    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-run-netns\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-kubelet\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196791    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d640dee1-af4a-450f-8d66-924e7d08dd98-ovnkube-script-lib\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196794    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-kubelet\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196821    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9grbh\" (UniqueName: \"kubernetes.io/projected/d640dee1-af4a-450f-8d66-924e7d08dd98-kube-api-access-9grbh\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196854    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-var-lib-cni-bin\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196857    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3a23d815-2cef-43ba-91f1-56e9fe83a282-etc-selinux\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196860    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-host-run-netns\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196892    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh888\" (UniqueName: \"kubernetes.io/projected/943739a7-45a0-444f-86dd-f0f5722ee83d-kube-api-access-sh888\") pod \"node-resolver-54nt6\" (UID: \"943739a7-45a0-444f-86dd-f0f5722ee83d\") " pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196910    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-host-var-lib-cni-bin\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.196947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-systemd-units\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.197045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d640dee1-af4a-450f-8d66-924e7d08dd98-systemd-units\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.197089    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d640dee1-af4a-450f-8d66-924e7d08dd98-ovnkube-config\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.197198    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-multus-daemon-config\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.198832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.197776    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e7807e6d-3df2-4084-a6ea-a05859dd6eda-agent-certs\") pod \"konnectivity-agent-hm6m9\" (UID: \"e7807e6d-3df2-4084-a6ea-a05859dd6eda\") " pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:11.199479 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.199003    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d640dee1-af4a-450f-8d66-924e7d08dd98-ovn-node-metrics-cert\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.201400 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.201369    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:11.201400 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.201397    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:11.201568 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.201412    2559 projected.go:196] Error preparing data for projected volume kube-api-access-xtn5k for pod openshift-network-diagnostics/network-check-target-pmgnx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:11.201568 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.201484    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k podName:8dc6d460-fedb-4a07-89c0-3a16a76ed30b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:11.701465487 +0000 UTC m=+3.025987585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xtn5k" (UniqueName: "kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k") pod "network-check-target-pmgnx" (UID: "8dc6d460-fedb-4a07-89c0-3a16a76ed30b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:11.203973 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.203944    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5649c\" (UniqueName: \"kubernetes.io/projected/e3ff12cd-3065-46f6-a36b-dab33b9c16eb-kube-api-access-5649c\") pod \"multus-4x2kk\" (UID: \"e3ff12cd-3065-46f6-a36b-dab33b9c16eb\") " pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.204147 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.204122    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skqgc\" (UniqueName: \"kubernetes.io/projected/3a23d815-2cef-43ba-91f1-56e9fe83a282-kube-api-access-skqgc\") pod \"aws-ebs-csi-driver-node-thdqw\" (UID: \"3a23d815-2cef-43ba-91f1-56e9fe83a282\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.204906 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.204887    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcn69\" (UniqueName: \"kubernetes.io/projected/fb8e778a-bb91-4e9a-b7bb-d948def35021-kube-api-access-xcn69\") pod \"multus-additional-cni-plugins-xq4cx\" (UID: \"fb8e778a-bb91-4e9a-b7bb-d948def35021\") " pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.205500 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.205477    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9grbh\" (UniqueName: \"kubernetes.io/projected/d640dee1-af4a-450f-8d66-924e7d08dd98-kube-api-access-9grbh\") pod \"ovnkube-node-7pqql\" (UID: \"d640dee1-af4a-450f-8d66-924e7d08dd98\") " pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.205592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.205512    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npkwz\" (UniqueName: \"kubernetes.io/projected/978ce3f4-995f-41dc-a234-d77304a2a87d-kube-api-access-npkwz\") pod \"iptables-alerter-8j8xj\" (UID: \"978ce3f4-995f-41dc-a234-d77304a2a87d\") " pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.205962 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.205940    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh888\" (UniqueName: \"kubernetes.io/projected/943739a7-45a0-444f-86dd-f0f5722ee83d-kube-api-access-sh888\") pod \"node-resolver-54nt6\" (UID: \"943739a7-45a0-444f-86dd-f0f5722ee83d\") " pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.299336 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.299252    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qmc5z"
Jul 26 18:12:11.307972 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.307944    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5bt9b"
Jul 26 18:12:11.319705 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.319679    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8j8xj"
Jul 26 18:12:11.327595 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.327572    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:11.333505 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.333478    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-54nt6"
Jul 26 18:12:11.341391 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.341363    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:11.346066 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.346033    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:12:11.349535 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.349497    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw"
Jul 26 18:12:11.357496 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.357468    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4x2kk"
Jul 26 18:12:11.363388 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.363359    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xq4cx"
Jul 26 18:12:11.598714 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.598609    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:11.598714 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.598705    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:11.598941 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.598731    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:11.598941 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.598799    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs podName:618d2dc6-77cc-49e1-a892-b3f1f0a4eea4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:12.598780574 +0000 UTC m=+3.923302663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs") pod "network-metrics-daemon-jlkb4" (UID: "618d2dc6-77cc-49e1-a892-b3f1f0a4eea4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:11.598941 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.598817    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:11.598941 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.598868    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret podName:b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:12.598851857 +0000 UTC m=+3.923373946 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret") pod "global-pull-secret-syncer-n4ng2" (UID: "b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:11.800455 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:11.800421    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:11.800638 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.800587    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:11.800638 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.800608    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:11.800638 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.800632    2559 projected.go:196] Error preparing data for projected volume kube-api-access-xtn5k for pod openshift-network-diagnostics/network-check-target-pmgnx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:11.800760 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:11.800688    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k podName:8dc6d460-fedb-4a07-89c0-3a16a76ed30b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:12.800668431 +0000 UTC m=+4.125190525 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtn5k" (UniqueName: "kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k") pod "network-check-target-pmgnx" (UID: "8dc6d460-fedb-4a07-89c0-3a16a76ed30b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:12.025506 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.025459    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 18:07:11 +0000 UTC" deadline="2028-05-09 00:56:36.645407764 +0000 UTC"
Jul 26 18:12:12.025506 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.025492    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15654h44m24.61991877s"
Jul 26 18:12:12.052366 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:12.052326    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74fb6e83_4639_444c_8414_f7f7d07d9ddb.slice/crio-b0a00baecefa742d901ccd555f809d6b54dd0945e96b2e63678f395925413857 WatchSource:0}: Error finding container b0a00baecefa742d901ccd555f809d6b54dd0945e96b2e63678f395925413857: Status 404 returned error can't find the container with id b0a00baecefa742d901ccd555f809d6b54dd0945e96b2e63678f395925413857
Jul 26 18:12:12.054855 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:12.054576    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7807e6d_3df2_4084_a6ea_a05859dd6eda.slice/crio-f29b3fdad740b34616dc4ab42a7f57dfa20d9179b954fa9d31866be0172f3963 WatchSource:0}: Error finding container f29b3fdad740b34616dc4ab42a7f57dfa20d9179b954fa9d31866be0172f3963: Status 404 returned error can't find the container with id f29b3fdad740b34616dc4ab42a7f57dfa20d9179b954fa9d31866be0172f3963
Jul 26 18:12:12.056207 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:12.056100    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978ce3f4_995f_41dc_a234_d77304a2a87d.slice/crio-97d01b66252699fd89759123f38417495f72260ec0309bf1b5f7d03d7b06ef49 WatchSource:0}: Error finding container 97d01b66252699fd89759123f38417495f72260ec0309bf1b5f7d03d7b06ef49: Status 404 returned error can't find the container with id 97d01b66252699fd89759123f38417495f72260ec0309bf1b5f7d03d7b06ef49
Jul 26 18:12:12.056907 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:12.056886    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943739a7_45a0_444f_86dd_f0f5722ee83d.slice/crio-2a0ea999ed2e4bcce203c4c3c76629164942d74800d2eb45c01a05106bc5cbe3 WatchSource:0}: Error finding container 2a0ea999ed2e4bcce203c4c3c76629164942d74800d2eb45c01a05106bc5cbe3: Status 404 returned error can't find the container with id 2a0ea999ed2e4bcce203c4c3c76629164942d74800d2eb45c01a05106bc5cbe3
Jul 26 18:12:12.078943 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:12.078906    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a23d815_2cef_43ba_91f1_56e9fe83a282.slice/crio-bf7035718956fcb218790afa1c5fe72966de134af2236064646dc39ceaa6c3f0 WatchSource:0}: Error finding container bf7035718956fcb218790afa1c5fe72966de134af2236064646dc39ceaa6c3f0: Status 404 returned error can't find the container with id bf7035718956fcb218790afa1c5fe72966de134af2236064646dc39ceaa6c3f0
Jul 26 18:12:12.080045 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:12.080020    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd640dee1_af4a_450f_8d66_924e7d08dd98.slice/crio-f8fd48300a2917dad8c8655c17eb1a0222f3cc0242d1845aee3b2766535de70e WatchSource:0}: Error finding container f8fd48300a2917dad8c8655c17eb1a0222f3cc0242d1845aee3b2766535de70e: Status 404 returned error can't find the container with id f8fd48300a2917dad8c8655c17eb1a0222f3cc0242d1845aee3b2766535de70e
Jul 26 18:12:12.081252 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:12.081223    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb8e778a_bb91_4e9a_b7bb_d948def35021.slice/crio-a64f13c19fa6ed130f0e8f513ef1a162e81f794d14ae9a332d5151d5c50e663a WatchSource:0}: Error finding container a64f13c19fa6ed130f0e8f513ef1a162e81f794d14ae9a332d5151d5c50e663a: Status 404 returned error can't find the container with id a64f13c19fa6ed130f0e8f513ef1a162e81f794d14ae9a332d5151d5c50e663a
Jul 26 18:12:12.082395 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:12.082371    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3ff12cd_3065_46f6_a36b_dab33b9c16eb.slice/crio-50e6e27a34467012728b30422514e6eea854d881d8ae6cc895d58292e72f3c1f WatchSource:0}: Error finding container 50e6e27a34467012728b30422514e6eea854d881d8ae6cc895d58292e72f3c1f: Status 404 returned error can't find the container with id 50e6e27a34467012728b30422514e6eea854d881d8ae6cc895d58292e72f3c1f
Jul 26 18:12:12.083430 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:12.083408    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a8bada_5aff_496e_ad10_c359a3e6e416.slice/crio-8874243a23287ca8b45a3d381550c072ebd34d632dca842ae6f17ec14f75cee5 WatchSource:0}: Error finding container 8874243a23287ca8b45a3d381550c072ebd34d632dca842ae6f17ec14f75cee5: Status 404 returned error can't find the container with id 8874243a23287ca8b45a3d381550c072ebd34d632dca842ae6f17ec14f75cee5
Jul 26 18:12:12.170291 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.170085    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:12.170414 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:12.170383    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:12.179564 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.179532    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8j8xj" event={"ID":"978ce3f4-995f-41dc-a234-d77304a2a87d","Type":"ContainerStarted","Data":"97d01b66252699fd89759123f38417495f72260ec0309bf1b5f7d03d7b06ef49"}
Jul 26 18:12:12.180835 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.180782    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hm6m9" event={"ID":"e7807e6d-3df2-4084-a6ea-a05859dd6eda","Type":"ContainerStarted","Data":"f29b3fdad740b34616dc4ab42a7f57dfa20d9179b954fa9d31866be0172f3963"}
Jul 26 18:12:12.182901 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.182874    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal" event={"ID":"a2a497afdef47058f3109e35eaa2548a","Type":"ContainerStarted","Data":"03c3a84fbe229f2dd31466d488052d58546a08c07c794445df750d0f40940d06"}
Jul 26 18:12:12.184244 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.184219    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4x2kk" event={"ID":"e3ff12cd-3065-46f6-a36b-dab33b9c16eb","Type":"ContainerStarted","Data":"50e6e27a34467012728b30422514e6eea854d881d8ae6cc895d58292e72f3c1f"}
Jul 26 18:12:12.185384 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.185362    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw" event={"ID":"3a23d815-2cef-43ba-91f1-56e9fe83a282","Type":"ContainerStarted","Data":"bf7035718956fcb218790afa1c5fe72966de134af2236064646dc39ceaa6c3f0"}
Jul 26 18:12:12.186463 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.186441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-54nt6" event={"ID":"943739a7-45a0-444f-86dd-f0f5722ee83d","Type":"ContainerStarted","Data":"2a0ea999ed2e4bcce203c4c3c76629164942d74800d2eb45c01a05106bc5cbe3"}
Jul 26 18:12:12.187454 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.187431    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qmc5z" event={"ID":"74fb6e83-4639-444c-8414-f7f7d07d9ddb","Type":"ContainerStarted","Data":"b0a00baecefa742d901ccd555f809d6b54dd0945e96b2e63678f395925413857"}
Jul 26 18:12:12.188545 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.188518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5bt9b" event={"ID":"35a8bada-5aff-496e-ad10-c359a3e6e416","Type":"ContainerStarted","Data":"8874243a23287ca8b45a3d381550c072ebd34d632dca842ae6f17ec14f75cee5"}
Jul 26 18:12:12.189699 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.189656    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq4cx" event={"ID":"fb8e778a-bb91-4e9a-b7bb-d948def35021","Type":"ContainerStarted","Data":"a64f13c19fa6ed130f0e8f513ef1a162e81f794d14ae9a332d5151d5c50e663a"}
Jul 26 18:12:12.191020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.191002    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerStarted","Data":"f8fd48300a2917dad8c8655c17eb1a0222f3cc0242d1845aee3b2766535de70e"}
Jul 26 18:12:12.197444 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.197402    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-70.ec2.internal" podStartSLOduration=2.197386269 podStartE2EDuration="2.197386269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:12.197033319 +0000 UTC m=+3.521555419" watchObservedRunningTime="2026-07-26 18:12:12.197386269 +0000 UTC m=+3.521908367"
Jul 26 18:12:12.607818 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.607730    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:12.607818 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.607810    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:12.608040 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:12.607949    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:12.608040 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:12.608010    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs podName:618d2dc6-77cc-49e1-a892-b3f1f0a4eea4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:14.607991774 +0000 UTC m=+5.932513864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs") pod "network-metrics-daemon-jlkb4" (UID: "618d2dc6-77cc-49e1-a892-b3f1f0a4eea4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:12.608429 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:12.608409    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:12.608510 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:12.608468    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret podName:b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:14.608452778 +0000 UTC m=+5.932974870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret") pod "global-pull-secret-syncer-n4ng2" (UID: "b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:12.750247 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.749686    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 26 18:12:12.809774 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:12.809429    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:12.809774 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:12.809703    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:12.809774 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:12.809727    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:12.809774 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:12.809740    2559 projected.go:196] Error preparing data for projected volume kube-api-access-xtn5k for pod openshift-network-diagnostics/network-check-target-pmgnx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:12.810092 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:12.809806    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k podName:8dc6d460-fedb-4a07-89c0-3a16a76ed30b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:14.809787873 +0000 UTC m=+6.134309948 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtn5k" (UniqueName: "kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k") pod "network-check-target-pmgnx" (UID: "8dc6d460-fedb-4a07-89c0-3a16a76ed30b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:13.026325 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:13.026259    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-25 18:07:11 +0000 UTC" deadline="2028-02-21 21:30:06.204928304 +0000 UTC"
Jul 26 18:12:13.026325 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:13.026323    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13803h17m53.178609097s"
Jul 26 18:12:13.171786 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:13.171749    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:13.171952 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:13.171771    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:13.171952 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:13.171936    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:13.172142 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:13.171962    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:13.214530 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:13.214492    2559 generic.go:356] "Generic (PLEG): container finished" podID="a0771439e9202664f9575f6e0ac07a38" containerID="ef7680334a98fa0b42fb5dad53a2a08573c6a5813637fc478037ea2cc27a7cfd" exitCode=0
Jul 26 18:12:13.215088 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:13.215060    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal" event={"ID":"a0771439e9202664f9575f6e0ac07a38","Type":"ContainerDied","Data":"ef7680334a98fa0b42fb5dad53a2a08573c6a5813637fc478037ea2cc27a7cfd"}
Jul 26 18:12:14.172193 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:14.171419    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:14.172193 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:14.171584    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:14.229154 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:14.228406    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal" event={"ID":"a0771439e9202664f9575f6e0ac07a38","Type":"ContainerStarted","Data":"c2356c3b99dc85c4029dc6631a1ce519677da22f16b6d0ccd258c2f2d66f9e3f"}
Jul 26 18:12:14.622966 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:14.622066    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:14.622966 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:14.622161    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:14.622966 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:14.622317    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:14.622966 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:14.622386    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret podName:b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:18.622366132 +0000 UTC m=+9.946888220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret") pod "global-pull-secret-syncer-n4ng2" (UID: "b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:14.622966 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:14.622803    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:14.622966 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:14.622857    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs podName:618d2dc6-77cc-49e1-a892-b3f1f0a4eea4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:18.622840032 +0000 UTC m=+9.947362111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs") pod "network-metrics-daemon-jlkb4" (UID: "618d2dc6-77cc-49e1-a892-b3f1f0a4eea4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:14.823191 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:14.823137    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:14.823394 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:14.823306    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:14.823394 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:14.823332    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:14.823394 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:14.823346    2559 projected.go:196] Error preparing data for projected volume kube-api-access-xtn5k for pod openshift-network-diagnostics/network-check-target-pmgnx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:14.823562 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:14.823417    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k podName:8dc6d460-fedb-4a07-89c0-3a16a76ed30b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:18.823397383 +0000 UTC m=+10.147919466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtn5k" (UniqueName: "kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k") pod "network-check-target-pmgnx" (UID: "8dc6d460-fedb-4a07-89c0-3a16a76ed30b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:15.170669 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:15.170631    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:15.170856 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:15.170679    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:15.170856 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:15.170786    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:15.171116 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:15.171075    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:16.170610 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:16.170573    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:16.171081 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:16.170728    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:17.170873 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:17.170835    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:17.171470 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:17.170842    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:17.171470 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:17.170988    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:17.171470 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:17.171030    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:18.170336 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:18.170299    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:18.170500 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:18.170436    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:18.656918 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:18.656787    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:18.656918 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:18.656863    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:18.657432 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:18.657004    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:18.657432 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:18.657077    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:18.657432 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:18.657081    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret podName:b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:26.657059735 +0000 UTC m=+17.981581832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret") pod "global-pull-secret-syncer-n4ng2" (UID: "b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:18.657432 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:18.657141    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs podName:618d2dc6-77cc-49e1-a892-b3f1f0a4eea4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:26.657124642 +0000 UTC m=+17.981646733 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs") pod "network-metrics-daemon-jlkb4" (UID: "618d2dc6-77cc-49e1-a892-b3f1f0a4eea4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:18.858392 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:18.858355    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:18.858633 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:18.858591    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:18.858752 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:18.858639    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:18.858752 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:18.858653    2559 projected.go:196] Error preparing data for projected volume kube-api-access-xtn5k for pod openshift-network-diagnostics/network-check-target-pmgnx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:18.858752 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:18.858728    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k podName:8dc6d460-fedb-4a07-89c0-3a16a76ed30b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:26.85870925 +0000 UTC m=+18.183231343 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtn5k" (UniqueName: "kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k") pod "network-check-target-pmgnx" (UID: "8dc6d460-fedb-4a07-89c0-3a16a76ed30b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:19.171707 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:19.171673    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:19.171707 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:19.171690    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:19.171939 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:19.171795    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:19.172002 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:19.171938    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:20.170832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:20.170797    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:20.171296 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:20.170930    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:21.170744 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:21.170712    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:21.170934 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:21.170716    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:21.170934 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:21.170842    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:21.170934 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:21.170904    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:22.170305 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:22.170276    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:22.170462 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:22.170409    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:23.170519 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:23.170480    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:23.171044 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:23.170534    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:23.171044 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:23.170639    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:23.171044 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:23.170747    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:24.170782 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:24.170742    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:24.171275 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:24.170880    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:25.170754 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:25.170719    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:25.170754 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:25.170746    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:25.171268 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:25.170850    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:25.171268 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:25.170961    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:26.170444 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:26.170409    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:26.170684 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:26.170517    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:26.721521 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:26.721484    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:26.721989 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:26.721567    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:26.721989 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:26.721673    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:26.721989 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:26.721735    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:26.721989 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:26.721753    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs podName:618d2dc6-77cc-49e1-a892-b3f1f0a4eea4 nodeName:}" failed. No retries permitted until 2026-07-26 18:12:42.721730844 +0000 UTC m=+34.046252931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs") pod "network-metrics-daemon-jlkb4" (UID: "618d2dc6-77cc-49e1-a892-b3f1f0a4eea4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:26.721989 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:26.721785    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret podName:b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:42.721768695 +0000 UTC m=+34.046290774 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret") pod "global-pull-secret-syncer-n4ng2" (UID: "b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:26.923306 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:26.923268    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:26.923492 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:26.923437    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:26.923492 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:26.923465    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:26.923492 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:26.923475    2559 projected.go:196] Error preparing data for projected volume kube-api-access-xtn5k for pod openshift-network-diagnostics/network-check-target-pmgnx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:26.923675 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:26.923527    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k podName:8dc6d460-fedb-4a07-89c0-3a16a76ed30b nodeName:}" failed. No retries permitted until 2026-07-26 18:12:42.923513198 +0000 UTC m=+34.248035276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtn5k" (UniqueName: "kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k") pod "network-check-target-pmgnx" (UID: "8dc6d460-fedb-4a07-89c0-3a16a76ed30b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:27.170469 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:27.170337    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:27.170469 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:27.170370    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:27.170737 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:27.170478    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:27.170737 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:27.170641    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:28.170415 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:28.170377    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:28.170919 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:28.170508    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:29.171067 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:29.171034    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:29.171545 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:29.171154    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:29.171545 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:29.171232    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:29.171545 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:29.171337    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:30.170731 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:30.170699    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:30.170899 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:30.170815    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:31.170870 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.170436    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:31.171572 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.170436    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:31.171572 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:31.170975    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:31.171572 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:31.171068    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:31.263793 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.263748    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-54nt6" event={"ID":"943739a7-45a0-444f-86dd-f0f5722ee83d","Type":"ContainerStarted","Data":"8c7a54ae576a40ac8de81a7866b96d735a4094bfa81b05853b8bbf4ffc76c36b"}
Jul 26 18:12:31.265267 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.265242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qmc5z" event={"ID":"74fb6e83-4639-444c-8414-f7f7d07d9ddb","Type":"ContainerStarted","Data":"5ff7accf75ff3dbb5ea7c6c456ed3b997f78ee9e8118e1f3316f655a9b8db480"}
Jul 26 18:12:31.266732 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.266704    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5bt9b" event={"ID":"35a8bada-5aff-496e-ad10-c359a3e6e416","Type":"ContainerStarted","Data":"a94a3be112aa6309537debe42db04ce923cfb4f8d76efad016b5934fcf000cd0"}
Jul 26 18:12:31.268542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.268491    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb8e778a-bb91-4e9a-b7bb-d948def35021" containerID="056dbe9d8fa41abc7d6899afd3c89900e0fcab7489054f2a737d9c16aa12bfcd" exitCode=0
Jul 26 18:12:31.268636 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.268587    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq4cx" event={"ID":"fb8e778a-bb91-4e9a-b7bb-d948def35021","Type":"ContainerDied","Data":"056dbe9d8fa41abc7d6899afd3c89900e0fcab7489054f2a737d9c16aa12bfcd"}
Jul 26 18:12:31.271239 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.271222    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:12:31.271568 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.271547    2559 generic.go:356] "Generic (PLEG): container finished" podID="d640dee1-af4a-450f-8d66-924e7d08dd98" containerID="dbc4f2857f4cd4fcf44f5be6315b0f6d164fd5c86da65c20f51d1fc30df70a57" exitCode=1
Jul 26 18:12:31.271668 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.271626    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerStarted","Data":"7b6b8e5fc31d0af6bcd5d239b18e831074dcbacf07da640b0ba201285536568d"}
Jul 26 18:12:31.271728 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.271701    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerStarted","Data":"45e102936b28c9d0664c055f69731a6399101f540fd1293c9f5aed3a684b4885"}
Jul 26 18:12:31.271780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.271757    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerStarted","Data":"206db968bd18ec16d7d7463ef1d2ccd7a00f184b7e0b05a08c7abeaaef56ba32"}
Jul 26 18:12:31.271855 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.271805    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerDied","Data":"dbc4f2857f4cd4fcf44f5be6315b0f6d164fd5c86da65c20f51d1fc30df70a57"}
Jul 26 18:12:31.271905 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.271854    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerStarted","Data":"13fa3bdf6dc756289e631550dec2341ab3d71efae819b0c1b5fe1fa111449f13"}
Jul 26 18:12:31.273167 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.273147    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hm6m9" event={"ID":"e7807e6d-3df2-4084-a6ea-a05859dd6eda","Type":"ContainerStarted","Data":"9b23d81f9339e23c63dbbd6d73d312fba2ad40024855180a6ce4af5e701aa62a"}
Jul 26 18:12:31.274660 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.274639    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4x2kk" event={"ID":"e3ff12cd-3065-46f6-a36b-dab33b9c16eb","Type":"ContainerStarted","Data":"729513d78b774a57f96e4a954571857d8e294b6cc1766f13e9e3c779742c2912"}
Jul 26 18:12:31.278589 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.278569    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw" event={"ID":"3a23d815-2cef-43ba-91f1-56e9fe83a282","Type":"ContainerStarted","Data":"4a9a748e48b1c49b1d6f129b4bfe48c66147f4639370a82c0d503b19321769ff"}
Jul 26 18:12:31.279353 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.279319    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-54nt6" podStartSLOduration=3.940788085 podStartE2EDuration="22.279307439s" totalImagesPullingTime="18.338519354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:31.278714156 +0000 UTC m=+22.603236252" watchObservedRunningTime="2026-07-26 18:12:31.279307439 +0000 UTC m=+22.603829533"
Jul 26 18:12:31.279406 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.279391    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-70.ec2.internal" podStartSLOduration=21.279385498 podStartE2EDuration="21.279385498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:14.244925081 +0000 UTC m=+5.569447190" watchObservedRunningTime="2026-07-26 18:12:31.279385498 +0000 UTC m=+22.603907595"
Jul 26 18:12:31.309223 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.309162    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hm6m9" podStartSLOduration=12.126444822 podStartE2EDuration="22.309140724s" totalImagesPullingTime="10.182695902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:31.308756583 +0000 UTC m=+22.633278734" watchObservedRunningTime="2026-07-26 18:12:31.309140724 +0000 UTC m=+22.633662824"
Jul 26 18:12:31.309414 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.309387    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5bt9b" podStartSLOduration=3.979105317 podStartE2EDuration="22.309376832s" totalImagesPullingTime="18.330271515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:31.293998776 +0000 UTC m=+22.618520872" watchObservedRunningTime="2026-07-26 18:12:31.309376832 +0000 UTC m=+22.633898930"
Jul 26 18:12:31.335122 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.335067    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4x2kk" podStartSLOduration=3.970683184 podStartE2EDuration="22.335052536s" totalImagesPullingTime="18.364369352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:31.334301287 +0000 UTC m=+22.658823384" watchObservedRunningTime="2026-07-26 18:12:31.335052536 +0000 UTC m=+22.659574633"
Jul 26 18:12:31.839688 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:31.839660    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 26 18:12:32.053346 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.053161    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-26T18:12:31.839679241Z","UUID":"65836063-3c58-4bb8-a772-45bba5161079","Handler":null,"Name":"","Endpoint":""}
Jul 26 18:12:32.054917 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.054895    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 26 18:12:32.055126 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.054937    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 26 18:12:32.170674 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.170630    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:32.170902 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:32.170757    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:32.284244 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.284211    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:12:32.284747 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.284719    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerStarted","Data":"c738a2858f100d8cd3daaaa9e13347306f7f1463205eb9ddc7d26cd63fe99390"}
Jul 26 18:12:32.286644 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.286595    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8j8xj" event={"ID":"978ce3f4-995f-41dc-a234-d77304a2a87d","Type":"ContainerStarted","Data":"12b8f4e8bc293d6dc7f45205d40a1f8ab5d81024f9c7ffbfbb15cf61f1f932ca"}
Jul 26 18:12:32.289377 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.289349    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw" event={"ID":"3a23d815-2cef-43ba-91f1-56e9fe83a282","Type":"ContainerStarted","Data":"6c66574d16570ca52b358c1b3705cbb997abb4dbfc5da03672f00159a90b27a3"}
Jul 26 18:12:32.319436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.319328    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qmc5z" podStartSLOduration=4.956838949 podStartE2EDuration="23.319305723s" totalImagesPullingTime="18.362466774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:31.382637575 +0000 UTC m=+22.707159664" watchObservedRunningTime="2026-07-26 18:12:32.319305723 +0000 UTC m=+23.643827823"
Jul 26 18:12:32.319738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:32.319557    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8j8xj" podStartSLOduration=4.981067223 podStartE2EDuration="23.319544847s" totalImagesPullingTime="18.338477624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:32.31926198 +0000 UTC m=+23.643784078" watchObservedRunningTime="2026-07-26 18:12:32.319544847 +0000 UTC m=+23.644066946"
Jul 26 18:12:33.170609 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:33.170572    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:33.170884 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:33.170579    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:33.170884 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:33.170741    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:33.170884 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:33.170809    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:33.223838 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:33.223803    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:33.224538 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:33.224519    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:33.293934 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:33.293897    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw" event={"ID":"3a23d815-2cef-43ba-91f1-56e9fe83a282","Type":"ContainerStarted","Data":"0cd7c511b924aa07203a969630cc0d4ee9637a9d759c7e4cf27a4f60ce6043b3"}
Jul 26 18:12:33.316546 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:33.316491    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-thdqw" podStartSLOduration=3.898230778 podStartE2EDuration="24.316470451s" totalImagesPullingTime="20.418239673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:33.316182094 +0000 UTC m=+24.640704197" watchObservedRunningTime="2026-07-26 18:12:33.316470451 +0000 UTC m=+24.640992549"
Jul 26 18:12:34.170476 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:34.170277    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:34.170730 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:34.170555    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:34.299789 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:34.299757    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:12:34.300228 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:34.300187    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerStarted","Data":"2fd0823a3e6025ec257b9d82438617d37746d83031a709e5bb243b412225af52"}
Jul 26 18:12:34.300284 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:34.300254    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 26 18:12:35.171092 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:35.171049    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:35.171311 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:35.171208    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:35.171311 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:35.171273    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:35.171450 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:35.171398    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:36.171159 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.170967    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:36.171869 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:36.171234    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:36.306560 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.306527    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb8e778a-bb91-4e9a-b7bb-d948def35021" containerID="6281a58db8ca49e16205127bf9e040e9a84a3c94000ad7d12b79b0c74d8ccb3b" exitCode=0
Jul 26 18:12:36.306749 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.306633    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq4cx" event={"ID":"fb8e778a-bb91-4e9a-b7bb-d948def35021","Type":"ContainerDied","Data":"6281a58db8ca49e16205127bf9e040e9a84a3c94000ad7d12b79b0c74d8ccb3b"}
Jul 26 18:12:36.310280 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.310264    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:12:36.310696 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.310667    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerStarted","Data":"a21eb2c5d64192caaabadeb7cee0f8853b1a1007c329200791ba6fd779c0dc58"}
Jul 26 18:12:36.311009 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.310990    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:36.311131 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.311117    2559 scope.go:117] "RemoveContainer" containerID="dbc4f2857f4cd4fcf44f5be6315b0f6d164fd5c86da65c20f51d1fc30df70a57"
Jul 26 18:12:36.328006 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.327981    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:36.559345 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.559266    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:36.559500 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.559435    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 26 18:12:36.559854 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:36.559832    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hm6m9"
Jul 26 18:12:37.170148 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:37.170091    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:37.170350 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:37.170254    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:37.170350 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:37.170290    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:37.170465 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:37.170383    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:37.316916 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:37.316845    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:12:37.317311 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:37.317240    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" event={"ID":"d640dee1-af4a-450f-8d66-924e7d08dd98","Type":"ContainerStarted","Data":"f1859519c3463378d9937dde31325b7209ba6e452defa6e36bf6014b2227f48f"}
Jul 26 18:12:37.317311 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:37.317282    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 26 18:12:37.317525 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:37.317502    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:37.333131 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:37.333096    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:37.349387 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:37.349334    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" podStartSLOduration=9.747497062 podStartE2EDuration="28.349318183s" totalImagesPullingTime="18.601821121s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:37.347320993 +0000 UTC m=+28.671843091" watchObservedRunningTime="2026-07-26 18:12:37.349318183 +0000 UTC m=+28.673840279"
Jul 26 18:12:38.170842 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:38.170804    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:38.171069 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:38.170928    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:38.321956 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:38.321919    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb8e778a-bb91-4e9a-b7bb-d948def35021" containerID="327d1803f2c0cd115ccee90e6ae9fb6a1f0f2947bcaef95eea9fea19c68bb626" exitCode=0
Jul 26 18:12:38.322322 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:38.322004    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq4cx" event={"ID":"fb8e778a-bb91-4e9a-b7bb-d948def35021","Type":"ContainerDied","Data":"327d1803f2c0cd115ccee90e6ae9fb6a1f0f2947bcaef95eea9fea19c68bb626"}
Jul 26 18:12:38.322322 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:38.322091    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 26 18:12:39.171452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:39.171417    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:39.171629 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:39.171524    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:39.171629 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:39.171571    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:39.171723 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:39.171664    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:39.324648 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:39.324607    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 26 18:12:40.170912 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:40.170873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:40.171119 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:40.170993    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:40.329582 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:40.329546    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb8e778a-bb91-4e9a-b7bb-d948def35021" containerID="5af5c471e3d2cbd777282d7510c70bc5678b778bf71bf7b57d6b0bfec6b482d1" exitCode=0
Jul 26 18:12:40.329970 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:40.329597    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq4cx" event={"ID":"fb8e778a-bb91-4e9a-b7bb-d948def35021","Type":"ContainerDied","Data":"5af5c471e3d2cbd777282d7510c70bc5678b778bf71bf7b57d6b0bfec6b482d1"}
Jul 26 18:12:40.865096 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:40.865060    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:12:40.865306 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:40.865292    2559 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 26 18:12:40.879233 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:40.879182    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" podUID="d640dee1-af4a-450f-8d66-924e7d08dd98" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 26 18:12:40.888076 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:40.888037    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql" podUID="d640dee1-af4a-450f-8d66-924e7d08dd98" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 26 18:12:41.170605 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:41.170569    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:41.170892 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:41.170569    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:41.170892 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:41.170719    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:41.170892 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:41.170756    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:42.170647 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:42.170600    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:42.171055 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:42.170733    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:42.743473 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:42.743434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:42.743680 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:42.743493    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:42.743680 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:42.743596    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:42.743680 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:42.743669    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs podName:618d2dc6-77cc-49e1-a892-b3f1f0a4eea4 nodeName:}" failed. No retries permitted until 2026-07-26 18:13:14.743653835 +0000 UTC m=+66.068175912 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs") pod "network-metrics-daemon-jlkb4" (UID: "618d2dc6-77cc-49e1-a892-b3f1f0a4eea4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 26 18:12:42.743832 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:42.743596    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:42.743832 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:42.743733    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret podName:b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b nodeName:}" failed. No retries permitted until 2026-07-26 18:13:14.743719986 +0000 UTC m=+66.068242062 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret") pod "global-pull-secret-syncer-n4ng2" (UID: "b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b") : object "kube-system"/"original-pull-secret" not registered
Jul 26 18:12:42.945738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:42.945700    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:42.945899 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:42.945869    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 26 18:12:42.945899 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:42.945891    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 26 18:12:42.946074 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:42.945908    2559 projected.go:196] Error preparing data for projected volume kube-api-access-xtn5k for pod openshift-network-diagnostics/network-check-target-pmgnx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:42.946074 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:42.945967    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k podName:8dc6d460-fedb-4a07-89c0-3a16a76ed30b nodeName:}" failed. No retries permitted until 2026-07-26 18:13:14.945950484 +0000 UTC m=+66.270472559 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtn5k" (UniqueName: "kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k") pod "network-check-target-pmgnx" (UID: "8dc6d460-fedb-4a07-89c0-3a16a76ed30b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 26 18:12:43.170945 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:43.170739    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:43.171364 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:43.170803    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:43.171364 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:43.171056    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:43.171364 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:43.171198    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:44.170390 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:44.170355    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:44.170575 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:44.170501    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:45.170075 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:45.170042    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:45.170489 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:45.170046    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:45.170489 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:45.170198    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:45.170489 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:45.170244    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:46.170681 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:46.170655    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:46.171052 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:46.170759    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:46.346872 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:46.346833    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb8e778a-bb91-4e9a-b7bb-d948def35021" containerID="2c5a8b9ebb0de1d5adc62dd8839f6347fcaa5037a2fe2bb27ff07efc942505b4" exitCode=0
Jul 26 18:12:46.347041 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:46.346894    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq4cx" event={"ID":"fb8e778a-bb91-4e9a-b7bb-d948def35021","Type":"ContainerDied","Data":"2c5a8b9ebb0de1d5adc62dd8839f6347fcaa5037a2fe2bb27ff07efc942505b4"}
Jul 26 18:12:47.173912 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:47.173883    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:47.174288 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:47.173886    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:47.174288 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:47.174025    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:47.174288 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:47.174127    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:47.352645 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:47.352599    2559 generic.go:356] "Generic (PLEG): container finished" podID="fb8e778a-bb91-4e9a-b7bb-d948def35021" containerID="732232586f50b5b767421384568cd5d171023f8f91dd52af303b94eceec8b2ab" exitCode=0
Jul 26 18:12:47.352819 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:47.352655    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq4cx" event={"ID":"fb8e778a-bb91-4e9a-b7bb-d948def35021","Type":"ContainerDied","Data":"732232586f50b5b767421384568cd5d171023f8f91dd52af303b94eceec8b2ab"}
Jul 26 18:12:48.170290 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:48.170250    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:48.170472 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:48.170356    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:48.358092 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:48.358061    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xq4cx" event={"ID":"fb8e778a-bb91-4e9a-b7bb-d948def35021","Type":"ContainerStarted","Data":"e18200a5ae357bd0eaa6b3ca9b17634549b0994c57c19049f467ae74d568bb01"}
Jul 26 18:12:48.384472 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:48.384418    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xq4cx" podStartSLOduration=9.024899438 podStartE2EDuration="39.384402755s" totalImagesPullingTime="30.359503317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:12:48.383956441 +0000 UTC m=+39.708478540" watchObservedRunningTime="2026-07-26 18:12:48.384402755 +0000 UTC m=+39.708924852"
Jul 26 18:12:49.171692 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:49.171656    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:49.171692 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:49.171696    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:49.171907 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:49.171771    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:49.171907 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:49.171885    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:50.170203 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:50.170166    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:50.170581 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:50.170271    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:51.170373 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:51.170334    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:51.170769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:51.170381    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:51.170769 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:51.170461    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:51.170769 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:51.170566    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:52.170961 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:52.170923    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:52.171334 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:52.171023    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:53.174006 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:53.173980    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:53.174442 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:53.174076    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:53.174442 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:53.174165    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:53.174442 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:53.174256    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:53.640678 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:53.640642    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4ng2"]
Jul 26 18:12:53.640921 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:53.640801    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:53.640986 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:53.640915    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:53.643550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:53.643523    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pmgnx"]
Jul 26 18:12:53.643742 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:53.643726    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:53.643907 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:53.643882    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:53.644438 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:53.644398    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jlkb4"]
Jul 26 18:12:53.644557 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:53.644502    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:53.644811 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:53.644611    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:55.176401 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:55.176372    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:55.177128 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:55.176520    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:55.177128 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:55.176555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:55.177128 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:55.176559    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:55.177128 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:55.176667    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:55.177128 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:55.176748    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:57.173413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:57.173234    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:57.173829 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:57.173264    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:57.173829 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:57.173494    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n4ng2" podUID="b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b"
Jul 26 18:12:57.173829 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:57.173269    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:57.173829 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:57.173566    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-jlkb4" podUID="618d2dc6-77cc-49e1-a892-b3f1f0a4eea4"
Jul 26 18:12:57.173829 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:12:57.173673    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmgnx" podUID="8dc6d460-fedb-4a07-89c0-3a16a76ed30b"
Jul 26 18:12:58.479332 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.479307    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-70.ec2.internal" event="NodeReady"
Jul 26 18:12:58.479728 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.479451    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 26 18:12:58.533782 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.533740    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ndlbj"]
Jul 26 18:12:58.554935 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.554906    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lvz4w"]
Jul 26 18:12:58.555079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.555040    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.557859 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.557835    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 26 18:12:58.558009 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.557835    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 26 18:12:58.558238 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.558224    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-w6wpb\""
Jul 26 18:12:58.576922 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.576888    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ndlbj"]
Jul 26 18:12:58.576922 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.576926    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lvz4w"]
Jul 26 18:12:58.577095 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.576994    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lvz4w"
Jul 26 18:12:58.580063 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.580012    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 26 18:12:58.580177 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.580151    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 26 18:12:58.580366 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.580350    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rv5hp\""
Jul 26 18:12:58.580443 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.580428    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 26 18:12:58.635960 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.635923    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ktklq"]
Jul 26 18:12:58.661882 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.661849    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/525f9463-314f-402a-8f92-ea019bd5d1da-metrics-tls\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.662033 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.661914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525f9463-314f-402a-8f92-ea019bd5d1da-tmp-dir\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.662033 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.661965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkk8x\" (UniqueName: \"kubernetes.io/projected/525f9463-314f-402a-8f92-ea019bd5d1da-kube-api-access-wkk8x\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.662146 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.662028    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/525f9463-314f-402a-8f92-ea019bd5d1da-config-volume\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.662329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.662310    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktklq"]
Jul 26 18:12:58.662454 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.662441    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.665377 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.665344    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 26 18:12:58.665377 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.665366    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 26 18:12:58.665566 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.665404    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zll5p\""
Jul 26 18:12:58.665662 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.665643    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 26 18:12:58.665717 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.665656    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 26 18:12:58.762589 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762506    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f93b966c-2271-4877-abdd-5af88a37caea-crio-socket\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.762589 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762540    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525f9463-314f-402a-8f92-ea019bd5d1da-tmp-dir\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.762589 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762561    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkk8x\" (UniqueName: \"kubernetes.io/projected/525f9463-314f-402a-8f92-ea019bd5d1da-kube-api-access-wkk8x\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.762589 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762580    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f93b966c-2271-4877-abdd-5af88a37caea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.762850 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762699    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/525f9463-314f-402a-8f92-ea019bd5d1da-config-volume\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.762850 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762731    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08f03152-02ec-4d03-9cda-5fd6fb479ade-cert\") pod \"ingress-canary-lvz4w\" (UID: \"08f03152-02ec-4d03-9cda-5fd6fb479ade\") " pod="openshift-ingress-canary/ingress-canary-lvz4w"
Jul 26 18:12:58.762850 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762761    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvlzg\" (UniqueName: \"kubernetes.io/projected/08f03152-02ec-4d03-9cda-5fd6fb479ade-kube-api-access-fvlzg\") pod \"ingress-canary-lvz4w\" (UID: \"08f03152-02ec-4d03-9cda-5fd6fb479ade\") " pod="openshift-ingress-canary/ingress-canary-lvz4w"
Jul 26 18:12:58.762850 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762806    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/525f9463-314f-402a-8f92-ea019bd5d1da-metrics-tls\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.762850 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762825    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93b966c-2271-4877-abdd-5af88a37caea-data-volume\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.762850 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762840    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbqgq\" (UniqueName: \"kubernetes.io/projected/f93b966c-2271-4877-abdd-5af88a37caea-kube-api-access-pbqgq\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.763064 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f93b966c-2271-4877-abdd-5af88a37caea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.763064 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.762924    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/525f9463-314f-402a-8f92-ea019bd5d1da-tmp-dir\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.763422 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.763399    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/525f9463-314f-402a-8f92-ea019bd5d1da-config-volume\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.767068 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.767042    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/525f9463-314f-402a-8f92-ea019bd5d1da-metrics-tls\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.771184 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.771161    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkk8x\" (UniqueName: \"kubernetes.io/projected/525f9463-314f-402a-8f92-ea019bd5d1da-kube-api-access-wkk8x\") pod \"dns-default-ndlbj\" (UID: \"525f9463-314f-402a-8f92-ea019bd5d1da\") " pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.863450 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.863414    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f93b966c-2271-4877-abdd-5af88a37caea-crio-socket\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.863647 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.863464    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f93b966c-2271-4877-abdd-5af88a37caea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.863694 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.863665    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f93b966c-2271-4877-abdd-5af88a37caea-crio-socket\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.863694 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.863667    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08f03152-02ec-4d03-9cda-5fd6fb479ade-cert\") pod \"ingress-canary-lvz4w\" (UID: \"08f03152-02ec-4d03-9cda-5fd6fb479ade\") " pod="openshift-ingress-canary/ingress-canary-lvz4w"
Jul 26 18:12:58.863753 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.863731    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvlzg\" (UniqueName: \"kubernetes.io/projected/08f03152-02ec-4d03-9cda-5fd6fb479ade-kube-api-access-fvlzg\") pod \"ingress-canary-lvz4w\" (UID: \"08f03152-02ec-4d03-9cda-5fd6fb479ade\") " pod="openshift-ingress-canary/ingress-canary-lvz4w"
Jul 26 18:12:58.863793 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.863783    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93b966c-2271-4877-abdd-5af88a37caea-data-volume\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.863844 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.863820    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbqgq\" (UniqueName: \"kubernetes.io/projected/f93b966c-2271-4877-abdd-5af88a37caea-kube-api-access-pbqgq\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.863892 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.863847    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f93b966c-2271-4877-abdd-5af88a37caea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.864170 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.864139    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f93b966c-2271-4877-abdd-5af88a37caea-data-volume\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.864494 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.864470    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f93b966c-2271-4877-abdd-5af88a37caea-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.864607 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.864593    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:12:58.865893 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.865869    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f93b966c-2271-4877-abdd-5af88a37caea-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.866034 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.866018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/08f03152-02ec-4d03-9cda-5fd6fb479ade-cert\") pod \"ingress-canary-lvz4w\" (UID: \"08f03152-02ec-4d03-9cda-5fd6fb479ade\") " pod="openshift-ingress-canary/ingress-canary-lvz4w"
Jul 26 18:12:58.874793 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.874762    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbqgq\" (UniqueName: \"kubernetes.io/projected/f93b966c-2271-4877-abdd-5af88a37caea-kube-api-access-pbqgq\") pod \"insights-runtime-extractor-ktklq\" (UID: \"f93b966c-2271-4877-abdd-5af88a37caea\") " pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:58.874903 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.874830    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvlzg\" (UniqueName: \"kubernetes.io/projected/08f03152-02ec-4d03-9cda-5fd6fb479ade-kube-api-access-fvlzg\") pod \"ingress-canary-lvz4w\" (UID: \"08f03152-02ec-4d03-9cda-5fd6fb479ade\") " pod="openshift-ingress-canary/ingress-canary-lvz4w"
Jul 26 18:12:58.886762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.886729    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lvz4w"
Jul 26 18:12:58.971022 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:58.970990    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ktklq"
Jul 26 18:12:59.075315 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.075279    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lvz4w"]
Jul 26 18:12:59.079552 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:59.079507    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f03152_02ec_4d03_9cda_5fd6fb479ade.slice/crio-16271fa73b0f78dd2aa65e75a9f97b4acd9fcaae5737f85bc2fa519c75870645 WatchSource:0}: Error finding container 16271fa73b0f78dd2aa65e75a9f97b4acd9fcaae5737f85bc2fa519c75870645: Status 404 returned error can't find the container with id 16271fa73b0f78dd2aa65e75a9f97b4acd9fcaae5737f85bc2fa519c75870645
Jul 26 18:12:59.081056 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.081028    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ndlbj"]
Jul 26 18:12:59.085998 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:59.085972    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod525f9463_314f_402a_8f92_ea019bd5d1da.slice/crio-9d73f927f99d053e07b57c134ee1e78e8610b8fb1c968b0112e7425bd45747bf WatchSource:0}: Error finding container 9d73f927f99d053e07b57c134ee1e78e8610b8fb1c968b0112e7425bd45747bf: Status 404 returned error can't find the container with id 9d73f927f99d053e07b57c134ee1e78e8610b8fb1c968b0112e7425bd45747bf
Jul 26 18:12:59.155558 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.155382    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ktklq"]
Jul 26 18:12:59.159075 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:12:59.159049    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93b966c_2271_4877_abdd_5af88a37caea.slice/crio-9f3860b2613d1b168414bcd0bd82bd9d442abdfc640fe9552f1a945871716a32 WatchSource:0}: Error finding container 9f3860b2613d1b168414bcd0bd82bd9d442abdfc640fe9552f1a945871716a32: Status 404 returned error can't find the container with id 9f3860b2613d1b168414bcd0bd82bd9d442abdfc640fe9552f1a945871716a32
Jul 26 18:12:59.173948 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.173925    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:12:59.174074 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.173926    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:12:59.174074 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.174035    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:12:59.176860 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.176836    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 18:12:59.176995 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.176838    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 18:12:59.176995 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.176844    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vf5dv\""
Jul 26 18:12:59.177111 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.176957    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7f4l\""
Jul 26 18:12:59.177111 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.176848    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 18:12:59.177111 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.176848    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 18:12:59.385078 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.384981    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktklq" event={"ID":"f93b966c-2271-4877-abdd-5af88a37caea","Type":"ContainerStarted","Data":"9f3860b2613d1b168414bcd0bd82bd9d442abdfc640fe9552f1a945871716a32"}
Jul 26 18:12:59.385998 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.385979    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lvz4w" event={"ID":"08f03152-02ec-4d03-9cda-5fd6fb479ade","Type":"ContainerStarted","Data":"16271fa73b0f78dd2aa65e75a9f97b4acd9fcaae5737f85bc2fa519c75870645"}
Jul 26 18:12:59.386989 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.386970    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ndlbj" event={"ID":"525f9463-314f-402a-8f92-ea019bd5d1da","Type":"ContainerStarted","Data":"9d73f927f99d053e07b57c134ee1e78e8610b8fb1c968b0112e7425bd45747bf"}
Jul 26 18:12:59.811307 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.811095    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5bdb77c7-5rp4n"]
Jul 26 18:12:59.820441 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.820410    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:12:59.824009 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.823981    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 26 18:12:59.825498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.825476    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 26 18:12:59.825757 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.825723    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 26 18:12:59.825951 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.825933    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 26 18:12:59.826246 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.826225    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5q42r\""
Jul 26 18:12:59.826434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.826418    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 26 18:12:59.826662 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.826646    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 26 18:12:59.826881 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.826864    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 26 18:12:59.829135 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.829109    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5bdb77c7-5rp4n"]
Jul 26 18:12:59.850936 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.850914    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 26 18:12:59.973632 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.973193    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-serving-cert\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:12:59.973632 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.973244    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-service-ca\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:12:59.973632 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.973293    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-console-config\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:12:59.973632 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.973347    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9wtp\" (UniqueName: \"kubernetes.io/projected/4cb916a1-34e5-491c-851d-51c0606f3f20-kube-api-access-g9wtp\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:12:59.973632 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.973415    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-oauth-serving-cert\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:12:59.973632 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.973442    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-oauth-config\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:12:59.973632 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:12:59.973469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-trusted-ca-bundle\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.075924 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.074701    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-oauth-serving-cert\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.075924 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.074757    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-oauth-config\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.075924 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.074782    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-trusted-ca-bundle\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.075924 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.074814    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-serving-cert\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.075924 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.074836    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-service-ca\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.075924 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.074877    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-console-config\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.075924 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.074929    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9wtp\" (UniqueName: \"kubernetes.io/projected/4cb916a1-34e5-491c-851d-51c0606f3f20-kube-api-access-g9wtp\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.078323 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.076844    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-oauth-serving-cert\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.078323 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.078085    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-console-config\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.078323 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.078321    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-trusted-ca-bundle\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.078548 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.078522    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-service-ca\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.086793 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.086763    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-serving-cert\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.093120 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.093047    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9wtp\" (UniqueName: \"kubernetes.io/projected/4cb916a1-34e5-491c-851d-51c0606f3f20-kube-api-access-g9wtp\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.094318 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.094256    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-oauth-config\") pod \"console-5c5bdb77c7-5rp4n\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") " pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.152981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.152632    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:00.350869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.350788    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5bdb77c7-5rp4n"]
Jul 26 18:13:00.361644 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:00.361592    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb916a1_34e5_491c_851d_51c0606f3f20.slice/crio-9bc8c28b8cb72abaaf93a7a6dd7abab1eea5e811bf6d3cf00a0b4be78fcc2a73 WatchSource:0}: Error finding container 9bc8c28b8cb72abaaf93a7a6dd7abab1eea5e811bf6d3cf00a0b4be78fcc2a73: Status 404 returned error can't find the container with id 9bc8c28b8cb72abaaf93a7a6dd7abab1eea5e811bf6d3cf00a0b4be78fcc2a73
Jul 26 18:13:00.392338 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:00.392285    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5bdb77c7-5rp4n" event={"ID":"4cb916a1-34e5-491c-851d-51c0606f3f20","Type":"ContainerStarted","Data":"9bc8c28b8cb72abaaf93a7a6dd7abab1eea5e811bf6d3cf00a0b4be78fcc2a73"}
Jul 26 18:13:01.427059 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.427028    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-44p5m"]
Jul 26 18:13:01.437835 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.437739    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.439344 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.439278    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-44p5m"]
Jul 26 18:13:01.441486 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.441290    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 26 18:13:01.441673 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.441611    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 26 18:13:01.441932 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.441864    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 26 18:13:01.441932 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.441887    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 26 18:13:01.442254 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.442090    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 26 18:13:01.442929 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.442905    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nf9lf\""
Jul 26 18:13:01.588022 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.587978    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7748711a-cb96-4c24-b83c-06b923d0a868-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.588287 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.588048    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7748711a-cb96-4c24-b83c-06b923d0a868-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.588287 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.588081    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7748711a-cb96-4c24-b83c-06b923d0a868-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.588287 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.588124    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wrr6\" (UniqueName: \"kubernetes.io/projected/7748711a-cb96-4c24-b83c-06b923d0a868-kube-api-access-8wrr6\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.690313 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.689121    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7748711a-cb96-4c24-b83c-06b923d0a868-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.690313 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.689216    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7748711a-cb96-4c24-b83c-06b923d0a868-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.690313 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.689248    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7748711a-cb96-4c24-b83c-06b923d0a868-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.690313 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.689287    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wrr6\" (UniqueName: \"kubernetes.io/projected/7748711a-cb96-4c24-b83c-06b923d0a868-kube-api-access-8wrr6\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.690313 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:13:01.689994    2559 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 26 18:13:01.690313 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:13:01.690065    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7748711a-cb96-4c24-b83c-06b923d0a868-prometheus-operator-tls podName:7748711a-cb96-4c24-b83c-06b923d0a868 nodeName:}" failed. No retries permitted until 2026-07-26 18:13:02.190042576 +0000 UTC m=+53.514564656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7748711a-cb96-4c24-b83c-06b923d0a868-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-44p5m" (UID: "7748711a-cb96-4c24-b83c-06b923d0a868") : secret "prometheus-operator-tls" not found
Jul 26 18:13:01.690851 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.690823    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7748711a-cb96-4c24-b83c-06b923d0a868-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.693236 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.693189    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7748711a-cb96-4c24-b83c-06b923d0a868-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:01.700758 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:01.700715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wrr6\" (UniqueName: \"kubernetes.io/projected/7748711a-cb96-4c24-b83c-06b923d0a868-kube-api-access-8wrr6\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:02.193786 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:02.193745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7748711a-cb96-4c24-b83c-06b923d0a868-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:02.196880 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:02.196855    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7748711a-cb96-4c24-b83c-06b923d0a868-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-44p5m\" (UID: \"7748711a-cb96-4c24-b83c-06b923d0a868\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:02.352098 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:02.352058    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m"
Jul 26 18:13:06.501986 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:06.501948    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-44p5m"]
Jul 26 18:13:06.505815 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:06.505786    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7748711a_cb96_4c24_b83c_06b923d0a868.slice/crio-29edc179ba9bd35ad4297bc60d15bb2855b4b422a8102ed662a268703090d959 WatchSource:0}: Error finding container 29edc179ba9bd35ad4297bc60d15bb2855b4b422a8102ed662a268703090d959: Status 404 returned error can't find the container with id 29edc179ba9bd35ad4297bc60d15bb2855b4b422a8102ed662a268703090d959
Jul 26 18:13:07.421403 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.420907    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5bdb77c7-5rp4n" event={"ID":"4cb916a1-34e5-491c-851d-51c0606f3f20","Type":"ContainerStarted","Data":"ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556"}
Jul 26 18:13:07.423242 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.423151    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktklq" event={"ID":"f93b966c-2271-4877-abdd-5af88a37caea","Type":"ContainerStarted","Data":"e6b8d38ae75954acd1c725c648fce4a4de6a0acc8baa4de772db66210f558b14"}
Jul 26 18:13:07.426307 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.425760    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lvz4w" event={"ID":"08f03152-02ec-4d03-9cda-5fd6fb479ade","Type":"ContainerStarted","Data":"02124526801cea1a7fc1c7346d5c3e730f358ace4863e32b13bf75fcddc9b7ef"}
Jul 26 18:13:07.428471 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.428407    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ndlbj" event={"ID":"525f9463-314f-402a-8f92-ea019bd5d1da","Type":"ContainerStarted","Data":"933d8044902a1629dc63164f2be6287e8610204a569977fe72eb911c4604fdcd"}
Jul 26 18:13:07.428471 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.428455    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ndlbj" event={"ID":"525f9463-314f-402a-8f92-ea019bd5d1da","Type":"ContainerStarted","Data":"0f336ccfd23900880360b5251e23f28cd30443bfdff509e8070d51e0835a41fd"}
Jul 26 18:13:07.429010 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.428988    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:13:07.430517 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.430489    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m" event={"ID":"7748711a-cb96-4c24-b83c-06b923d0a868","Type":"ContainerStarted","Data":"29edc179ba9bd35ad4297bc60d15bb2855b4b422a8102ed662a268703090d959"}
Jul 26 18:13:07.442179 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.441803    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c5bdb77c7-5rp4n" podStartSLOduration=2.393655786 podStartE2EDuration="8.441780106s" totalImagesPullingTime="6.04812432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:07.440787506 +0000 UTC m=+58.765309605" watchObservedRunningTime="2026-07-26 18:13:07.441780106 +0000 UTC m=+58.766302215"
Jul 26 18:13:07.460993 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.460927    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ndlbj" podStartSLOduration=3.239595154 podStartE2EDuration="9.460903969s" totalImagesPullingTime="6.221308815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:07.459476642 +0000 UTC m=+58.783998741" watchObservedRunningTime="2026-07-26 18:13:07.460903969 +0000 UTC m=+58.785426071"
Jul 26 18:13:07.477644 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:07.477579    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lvz4w" podStartSLOduration=2.219412526 podStartE2EDuration="9.477560387s" totalImagesPullingTime="7.258147861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:07.476798263 +0000 UTC m=+58.801320360" watchObservedRunningTime="2026-07-26 18:13:07.477560387 +0000 UTC m=+58.802082483"
Jul 26 18:13:10.153871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:10.153773    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:10.153871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:10.153847    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:10.159687 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:10.159659    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:10.444141 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:10.444111    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:10.890029 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:10.889946    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7pqql"
Jul 26 18:13:11.447117 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:11.447077    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m" event={"ID":"7748711a-cb96-4c24-b83c-06b923d0a868","Type":"ContainerStarted","Data":"abe1815735b325c6ffa90347b64d44bf3e03c323c5197d06ecad51ec32b2e759"}
Jul 26 18:13:11.447598 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:11.447143    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m" event={"ID":"7748711a-cb96-4c24-b83c-06b923d0a868","Type":"ContainerStarted","Data":"bd4bc2e943840bb7d6d3a0106bf96d1a4fecec3afef996640eccd36693d62b21"}
Jul 26 18:13:11.449262 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:11.449230    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktklq" event={"ID":"f93b966c-2271-4877-abdd-5af88a37caea","Type":"ContainerStarted","Data":"06ec51c8c0d27a743d4180c1805c1717ca6457ed101c8c8731001f8519841637"}
Jul 26 18:13:11.468326 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:11.468267    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-44p5m" podStartSLOduration=6.454194387 podStartE2EDuration="10.468245056s" totalImagesPullingTime="4.014050669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:11.468087619 +0000 UTC m=+62.792609717" watchObservedRunningTime="2026-07-26 18:13:11.468245056 +0000 UTC m=+62.792767161"
Jul 26 18:13:13.758030 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.757982    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"]
Jul 26 18:13:13.784411 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.784360    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"]
Jul 26 18:13:13.784411 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.784420    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"]
Jul 26 18:13:13.784657 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.784455    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:13.787388 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.787356    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 26 18:13:13.787803 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.787750    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-6q456\""
Jul 26 18:13:13.787974 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.787793    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 26 18:13:13.789470 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.789447    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:13.792148 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.792126    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 26 18:13:13.792267 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.792128    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 26 18:13:13.792495 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.792468    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 26 18:13:13.792601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.792498    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-4ggwr\""
Jul 26 18:13:13.798827 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.798805    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"]
Jul 26 18:13:13.803221 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.803196    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sp82r"]
Jul 26 18:13:13.808021 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.808000    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.811550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.811520    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 26 18:13:13.811550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.811537    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 26 18:13:13.811762 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.811578    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 26 18:13:13.811884 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.811866    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g7w8p\""
Jul 26 18:13:13.904298 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904264    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.904507 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904306    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmd4v\" (UniqueName: \"kubernetes.io/projected/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-kube-api-access-bmd4v\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:13.904507 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904333    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:13.904507 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904380    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7759bf11-07c1-4314-8133-74c3e58595fe-root\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.904507 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904412    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c47t5\" (UniqueName: \"kubernetes.io/projected/7759bf11-07c1-4314-8133-74c3e58595fe-kube-api-access-c47t5\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.904507 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzk7x\" (UniqueName: \"kubernetes.io/projected/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-api-access-dzk7x\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:13.904780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-wtmp\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.904780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904574    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:13.904780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904659    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:13.904780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904714    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:13.904780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7759bf11-07c1-4314-8133-74c3e58595fe-sys\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.904780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904766    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:13.905020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904794    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7759bf11-07c1-4314-8133-74c3e58595fe-metrics-client-ca\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.905020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904820    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:13.905020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.905020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904903    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-textfile\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.905020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904938    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-tls\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:13.905020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.904958    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:13.905020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:13.905014    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.005784 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.005745    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.005784 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.005784    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-textfile\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.005988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.005803    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-tls\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.005988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.005896    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.005988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.005915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.005988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.005938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.005988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.005956    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmd4v\" (UniqueName: \"kubernetes.io/projected/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-kube-api-access-bmd4v\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.005988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.005972    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.005988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.005991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7759bf11-07c1-4314-8133-74c3e58595fe-root\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.006302 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006022    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c47t5\" (UniqueName: \"kubernetes.io/projected/7759bf11-07c1-4314-8133-74c3e58595fe-kube-api-access-c47t5\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.006302 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006092    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7759bf11-07c1-4314-8133-74c3e58595fe-root\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.006302 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:13:14.005989    2559 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 26 18:13:14.006302 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:13:14.005994    2559 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 26 18:13:14.006302 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:13:14.006213    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-tls podName:7759bf11-07c1-4314-8133-74c3e58595fe nodeName:}" failed. No retries permitted until 2026-07-26 18:13:14.506190527 +0000 UTC m=+65.830712606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-tls") pod "node-exporter-sp82r" (UID: "7759bf11-07c1-4314-8133-74c3e58595fe") : secret "node-exporter-tls" not found
Jul 26 18:13:14.006302 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:13:14.006258    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-openshift-state-metrics-tls podName:63bf1d7f-f67b-4a56-a194-1cc4e0b891ee nodeName:}" failed. No retries permitted until 2026-07-26 18:13:14.506240203 +0000 UTC m=+65.830762283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-openshift-state-metrics-tls") pod "openshift-state-metrics-b8f49f5c6-lcnwr" (UID: "63bf1d7f-f67b-4a56-a194-1cc4e0b891ee") : secret "openshift-state-metrics-tls" not found
Jul 26 18:13:14.006602 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006365    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzk7x\" (UniqueName: \"kubernetes.io/projected/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-api-access-dzk7x\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.006602 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006396    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-wtmp\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.006602 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006423    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.006602 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006476    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.006602 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006546    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.006602 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006579    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7759bf11-07c1-4314-8133-74c3e58595fe-sys\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.006602 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-textfile\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.006973 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006608    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.006973 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006658    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.006973 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7759bf11-07c1-4314-8133-74c3e58595fe-metrics-client-ca\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.006973 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006733    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.006973 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.006834    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.007219 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.007135    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.007219 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.007201    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7759bf11-07c1-4314-8133-74c3e58595fe-metrics-client-ca\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.007361 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.007340    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-wtmp\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.007423 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.007404    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7759bf11-07c1-4314-8133-74c3e58595fe-sys\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.007423 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.007406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.007793 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.007768    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.008804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.008742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.008804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.008787    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.008926 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.008886    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.009714 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.009692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.015655 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.015610    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmd4v\" (UniqueName: \"kubernetes.io/projected/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-kube-api-access-bmd4v\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.016387 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.016366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzk7x\" (UniqueName: \"kubernetes.io/projected/26ef4de6-17cb-4fc3-93cf-2c3e724b3f49-kube-api-access-dzk7x\") pod \"kube-state-metrics-56d9f944b4-8rg26\" (UID: \"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.017146 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.017127    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c47t5\" (UniqueName: \"kubernetes.io/projected/7759bf11-07c1-4314-8133-74c3e58595fe-kube-api-access-c47t5\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.105100 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.105064    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"
Jul 26 18:13:14.241789 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.241760    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26"]
Jul 26 18:13:14.245184 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:14.245156    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26ef4de6_17cb_4fc3_93cf_2c3e724b3f49.slice/crio-fad5bdd4be66649f7faad9904dfea4f0e230844ed61ce9ae8681ed9dc3da76a5 WatchSource:0}: Error finding container fad5bdd4be66649f7faad9904dfea4f0e230844ed61ce9ae8681ed9dc3da76a5: Status 404 returned error can't find the container with id fad5bdd4be66649f7faad9904dfea4f0e230844ed61ce9ae8681ed9dc3da76a5
Jul 26 18:13:14.462480 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.462445    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ktklq" event={"ID":"f93b966c-2271-4877-abdd-5af88a37caea","Type":"ContainerStarted","Data":"22e3192f912f241cc84073e0e9e5967df010980fd6c602140076ad48723344c2"}
Jul 26 18:13:14.463632 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.463598    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26" event={"ID":"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49","Type":"ContainerStarted","Data":"fad5bdd4be66649f7faad9904dfea4f0e230844ed61ce9ae8681ed9dc3da76a5"}
Jul 26 18:13:14.494070 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.494019    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ktklq" podStartSLOduration=1.778737997 podStartE2EDuration="16.49400237s" totalImagesPullingTime="14.715264373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:14.492868523 +0000 UTC m=+65.817390619" watchObservedRunningTime="2026-07-26 18:13:14.49400237 +0000 UTC m=+65.818524467"
Jul 26 18:13:14.510384 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.510350    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-tls\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.510583 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.510389    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.512721 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.512700    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7759bf11-07c1-4314-8133-74c3e58595fe-node-exporter-tls\") pod \"node-exporter-sp82r\" (UID: \"7759bf11-07c1-4314-8133-74c3e58595fe\") " pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.512835 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.512816    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/63bf1d7f-f67b-4a56-a194-1cc4e0b891ee-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-lcnwr\" (UID: \"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.698006 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.697968    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"
Jul 26 18:13:14.717258 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.717174    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sp82r"
Jul 26 18:13:14.731211 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:14.731177    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7759bf11_07c1_4314_8133_74c3e58595fe.slice/crio-10c7a5122a4531e4464591a52ca0fa6c22d2ebb83006e85f9e38399ed04f6a9a WatchSource:0}: Error finding container 10c7a5122a4531e4464591a52ca0fa6c22d2ebb83006e85f9e38399ed04f6a9a: Status 404 returned error can't find the container with id 10c7a5122a4531e4464591a52ca0fa6c22d2ebb83006e85f9e38399ed04f6a9a
Jul 26 18:13:14.812912 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.812882    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:13:14.813270 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.812948    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:13:14.817340 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.817307    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 26 18:13:14.817570 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.817556    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 26 18:13:14.826113 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.826050    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b-original-pull-secret\") pod \"global-pull-secret-syncer-n4ng2\" (UID: \"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b\") " pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:13:14.826281 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.826239    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/618d2dc6-77cc-49e1-a892-b3f1f0a4eea4-metrics-certs\") pod \"network-metrics-daemon-jlkb4\" (UID: \"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4\") " pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:13:14.845062 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.845032    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr"]
Jul 26 18:13:14.848305 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:14.848274    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63bf1d7f_f67b_4a56_a194_1cc4e0b891ee.slice/crio-daaf6e350abc6e29cbc98458bca92fa0801ab9fcbca8e0b65bda4a5f4d822432 WatchSource:0}: Error finding container daaf6e350abc6e29cbc98458bca92fa0801ab9fcbca8e0b65bda4a5f4d822432: Status 404 returned error can't find the container with id daaf6e350abc6e29cbc98458bca92fa0801ab9fcbca8e0b65bda4a5f4d822432
Jul 26 18:13:14.984077 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.984042    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 18:13:14.989287 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.989258    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:14.992560 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.992364    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 26 18:13:14.992560 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.992418    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 26 18:13:14.992560 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.992427    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 26 18:13:14.992560 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.992506    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 26 18:13:14.992883 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.992636    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 26 18:13:14.992883 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.992821    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 26 18:13:14.992883 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.992839    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 26 18:13:14.993105 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.993092    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 26 18:13:14.993180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.993162    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-g26l5\""
Jul 26 18:13:14.993425 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:14.993380    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 26 18:13:15.007209 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.007175    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 18:13:15.015161 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015117    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015341 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-web-config\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015341 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015239    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3dd18ce3-429f-49d0-8397-2a369659fe31-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015341 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015286    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dd18ce3-429f-49d0-8397-2a369659fe31-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015361    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015394    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3dd18ce3-429f-49d0-8397-2a369659fe31-config-out\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015413    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n74j\" (UniqueName: \"kubernetes.io/projected/3dd18ce3-429f-49d0-8397-2a369659fe31-kube-api-access-6n74j\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015455    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015773 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015508    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dd18ce3-429f-49d0-8397-2a369659fe31-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015773 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015587    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:13:15.015773 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015663    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015773 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015713    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-config-volume\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.015773 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.016105 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.015776    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3dd18ce3-429f-49d0-8397-2a369659fe31-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.018735 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.018716    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 26 18:13:15.029193 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.029167    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 26 18:13:15.039240 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.039202    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtn5k\" (UniqueName: \"kubernetes.io/projected/8dc6d460-fedb-4a07-89c0-3a16a76ed30b-kube-api-access-xtn5k\") pod \"network-check-target-pmgnx\" (UID: \"8dc6d460-fedb-4a07-89c0-3a16a76ed30b\") " pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:13:15.088148 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.088103    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vf5dv\""
Jul 26 18:13:15.091769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.091737    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n4ng2"
Jul 26 18:13:15.095472 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.095445    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-jlkb4"
Jul 26 18:13:15.098379 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.098356    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7f4l\""
Jul 26 18:13:15.106088 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.106062    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:13:15.117117 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117085    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3dd18ce3-429f-49d0-8397-2a369659fe31-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117247 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117127    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dd18ce3-429f-49d0-8397-2a369659fe31-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117247 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117166    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117247 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117194    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3dd18ce3-429f-49d0-8397-2a369659fe31-config-out\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117247 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117218    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n74j\" (UniqueName: \"kubernetes.io/projected/3dd18ce3-429f-49d0-8397-2a369659fe31-kube-api-access-6n74j\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117247 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117246    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dd18ce3-429f-49d0-8397-2a369659fe31-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117303    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117337    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-config-volume\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3dd18ce3-429f-49d0-8397-2a369659fe31-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.117498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.117474    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-web-config\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.119794 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.118310    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dd18ce3-429f-49d0-8397-2a369659fe31-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.119794 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:13:15.119107    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3dd18ce3-429f-49d0-8397-2a369659fe31-alertmanager-trusted-ca-bundle podName:3dd18ce3-429f-49d0-8397-2a369659fe31 nodeName:}" failed. No retries permitted until 2026-07-26 18:13:15.619080857 +0000 UTC m=+66.943602949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/3dd18ce3-429f-49d0-8397-2a369659fe31-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "3dd18ce3-429f-49d0-8397-2a369659fe31") : configmap references non-existent config key: ca-bundle.crt
Jul 26 18:13:15.122794 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.120980    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3dd18ce3-429f-49d0-8397-2a369659fe31-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.122794 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.121092    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3dd18ce3-429f-49d0-8397-2a369659fe31-config-out\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.122794 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.121729    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.123936 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.123853    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.124891 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.124846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-config-volume\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.125431 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.125409    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.125838 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.125818    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-web-config\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.126145 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.126126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3dd18ce3-429f-49d0-8397-2a369659fe31-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.128881 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.128844    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n74j\" (UniqueName: \"kubernetes.io/projected/3dd18ce3-429f-49d0-8397-2a369659fe31-kube-api-access-6n74j\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.129241 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.129220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.129840 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.129816    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3dd18ce3-429f-49d0-8397-2a369659fe31-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.278084 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.278001    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n4ng2"]
Jul 26 18:13:15.306728 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.306597    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-jlkb4"]
Jul 26 18:13:15.324377 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.324301    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pmgnx"]
Jul 26 18:13:15.469404 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.469364    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sp82r" event={"ID":"7759bf11-07c1-4314-8133-74c3e58595fe","Type":"ContainerStarted","Data":"10c7a5122a4531e4464591a52ca0fa6c22d2ebb83006e85f9e38399ed04f6a9a"}
Jul 26 18:13:15.471853 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.471765    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr" event={"ID":"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee","Type":"ContainerStarted","Data":"5e67230713cb06d0c75581bf8e257211a6c24ec9938a0ab30e11a09c94107d0a"}
Jul 26 18:13:15.471853 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.471834    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr" event={"ID":"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee","Type":"ContainerStarted","Data":"55b206042026c867a8ea11bfaeadb516bad1d959e4a3b173cd49dd1f5ad644fe"}
Jul 26 18:13:15.472059 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.471865    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr" event={"ID":"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee","Type":"ContainerStarted","Data":"daaf6e350abc6e29cbc98458bca92fa0801ab9fcbca8e0b65bda4a5f4d822432"}
Jul 26 18:13:15.569369 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:15.569318    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod618d2dc6_77cc_49e1_a892_b3f1f0a4eea4.slice/crio-0a64733d7764bf49d82bfd9b26ca8120a53ff7398f8625502882968b23aa0028 WatchSource:0}: Error finding container 0a64733d7764bf49d82bfd9b26ca8120a53ff7398f8625502882968b23aa0028: Status 404 returned error can't find the container with id 0a64733d7764bf49d82bfd9b26ca8120a53ff7398f8625502882968b23aa0028
Jul 26 18:13:15.569982 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:15.569955    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb28a6ca5_8e3c_4d29_8ed5_a684e40f6f0b.slice/crio-978cf3e3fd9f65bbed7cb8bb4359cd210c25aef3d247b2f2156745caeb6398e5 WatchSource:0}: Error finding container 978cf3e3fd9f65bbed7cb8bb4359cd210c25aef3d247b2f2156745caeb6398e5: Status 404 returned error can't find the container with id 978cf3e3fd9f65bbed7cb8bb4359cd210c25aef3d247b2f2156745caeb6398e5
Jul 26 18:13:15.624157 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.624118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dd18ce3-429f-49d0-8397-2a369659fe31-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.625093 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.625065    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dd18ce3-429f-49d0-8397-2a369659fe31-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3dd18ce3-429f-49d0-8397-2a369659fe31\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.783403 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:15.783307    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc6d460_fedb_4a07_89c0_3a16a76ed30b.slice/crio-b999d53a3c1f3765f35c6f34dbedee2da8a767e3a8604284625df05a25c102d1 WatchSource:0}: Error finding container b999d53a3c1f3765f35c6f34dbedee2da8a767e3a8604284625df05a25c102d1: Status 404 returned error can't find the container with id b999d53a3c1f3765f35c6f34dbedee2da8a767e3a8604284625df05a25c102d1
Jul 26 18:13:15.886816 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.886788    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"]
Jul 26 18:13:15.904293 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.904260    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 26 18:13:15.912034 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.911954    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"]
Jul 26 18:13:15.912164 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.912076    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:15.915005 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.914980    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 26 18:13:15.915179 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.915159    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 26 18:13:15.915529 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.915403    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 26 18:13:15.915529 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.915433    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 26 18:13:15.915942 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.915825    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-enml6luqk37rb\""
Jul 26 18:13:15.916228 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.916205    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 26 18:13:15.916325 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:15.916208    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-44jzg\""
Jul 26 18:13:16.027702 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.027668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqm92\" (UniqueName: \"kubernetes.io/projected/4089ec57-7c50-4c25-9921-d575c935d3e2-kube-api-access-xqm92\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.027876 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.027722    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-grpc-tls\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.027876 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.027763    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4089ec57-7c50-4c25-9921-d575c935d3e2-metrics-client-ca\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.027876 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.027809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.027876 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.027848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.028050 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.027896    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-tls\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.028050 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.027949    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.028050 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.027985    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.129921 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.129886    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.130134 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.129938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.130134 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.129983    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqm92\" (UniqueName: \"kubernetes.io/projected/4089ec57-7c50-4c25-9921-d575c935d3e2-kube-api-access-xqm92\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.130134 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.130043    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-grpc-tls\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.130134 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.130079    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4089ec57-7c50-4c25-9921-d575c935d3e2-metrics-client-ca\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.130134 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.130118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.130485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.130154    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.130485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.130202    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-tls\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.132160 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.131247    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 26 18:13:16.132160 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.132106    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4089ec57-7c50-4c25-9921-d575c935d3e2-metrics-client-ca\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.135003 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.134973    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.145035 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.137342    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-tls\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.145035 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.137908    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-grpc-tls\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.145035 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:16.138313    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dd18ce3_429f_49d0_8397_2a369659fe31.slice/crio-402e29adbec4b92d84a504072488d32d3184aa495e67590572d8718e5e31933a WatchSource:0}: Error finding container 402e29adbec4b92d84a504072488d32d3184aa495e67590572d8718e5e31933a: Status 404 returned error can't find the container with id 402e29adbec4b92d84a504072488d32d3184aa495e67590572d8718e5e31933a
Jul 26 18:13:16.145035 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.140334    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.154010 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.149187    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.154010 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.149752    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4089ec57-7c50-4c25-9921-d575c935d3e2-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.154010 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.150488    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqm92\" (UniqueName: \"kubernetes.io/projected/4089ec57-7c50-4c25-9921-d575c935d3e2-kube-api-access-xqm92\") pod \"thanos-querier-8496b7b7c8-9dxhx\" (UID: \"4089ec57-7c50-4c25-9921-d575c935d3e2\") " pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.236093 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.236054    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:16.477577 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.477532    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jlkb4" event={"ID":"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4","Type":"ContainerStarted","Data":"0a64733d7764bf49d82bfd9b26ca8120a53ff7398f8625502882968b23aa0028"}
Jul 26 18:13:16.480706 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.479918    2559 generic.go:356] "Generic (PLEG): container finished" podID="7759bf11-07c1-4314-8133-74c3e58595fe" containerID="dc89fa05c16c082e8e1bec744b7ae0a019bb303a25fbab26cc18ab62de032d35" exitCode=0
Jul 26 18:13:16.480706 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.480308    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sp82r" event={"ID":"7759bf11-07c1-4314-8133-74c3e58595fe","Type":"ContainerDied","Data":"dc89fa05c16c082e8e1bec744b7ae0a019bb303a25fbab26cc18ab62de032d35"}
Jul 26 18:13:16.486380 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.486296    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pmgnx" event={"ID":"8dc6d460-fedb-4a07-89c0-3a16a76ed30b","Type":"ContainerStarted","Data":"b999d53a3c1f3765f35c6f34dbedee2da8a767e3a8604284625df05a25c102d1"}
Jul 26 18:13:16.488122 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.488049    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4ng2" event={"ID":"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b","Type":"ContainerStarted","Data":"978cf3e3fd9f65bbed7cb8bb4359cd210c25aef3d247b2f2156745caeb6398e5"}
Jul 26 18:13:16.493697 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.493215    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26" event={"ID":"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49","Type":"ContainerStarted","Data":"820584ce24ff82bf80b4195aa101a8bac4c16594d91a0e921ebce68b98ec6845"}
Jul 26 18:13:16.493697 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.493277    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26" event={"ID":"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49","Type":"ContainerStarted","Data":"ca7e582d8731c2cec820aae90ae7221b23d017e65d2dca956aefab8f166b51d9"}
Jul 26 18:13:16.493697 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.493308    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26" event={"ID":"26ef4de6-17cb-4fc3-93cf-2c3e724b3f49","Type":"ContainerStarted","Data":"e22afedca79062a8b34309f338cfd1fe32bc699a971a3f58cd24a9baec91353e"}
Jul 26 18:13:16.495191 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.495165    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dd18ce3-429f-49d0-8397-2a369659fe31","Type":"ContainerStarted","Data":"402e29adbec4b92d84a504072488d32d3184aa495e67590572d8718e5e31933a"}
Jul 26 18:13:16.523628 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:16.523556    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-8rg26" podStartSLOduration=1.932236404 podStartE2EDuration="3.5235334s" totalImagesPullingTime="1.591296996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:16.522860534 +0000 UTC m=+67.847382635" watchObservedRunningTime="2026-07-26 18:13:16.5235334 +0000 UTC m=+67.848055492"
Jul 26 18:13:17.507124 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:17.505392    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sp82r" event={"ID":"7759bf11-07c1-4314-8133-74c3e58595fe","Type":"ContainerStarted","Data":"9616bacc3db15d08f099ef4204ee1e395e7eb04304c2282933c9df67f04f39c5"}
Jul 26 18:13:17.510807 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:17.510779    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr" event={"ID":"63bf1d7f-f67b-4a56-a194-1cc4e0b891ee","Type":"ContainerStarted","Data":"0434d4f4bae8d8bef0d88a89afe0b05d4e34150947ba3a0b751dcdb8848f3c08"}
Jul 26 18:13:17.529999 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:17.529902    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"]
Jul 26 18:13:17.531313 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:17.530997    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-lcnwr" podStartSLOduration=2.156980001 podStartE2EDuration="4.530977064s" totalImagesPullingTime="2.373997063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:17.529958878 +0000 UTC m=+68.854480976" watchObservedRunningTime="2026-07-26 18:13:17.530977064 +0000 UTC m=+68.855499162"
Jul 26 18:13:18.179240 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.179202    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"]
Jul 26 18:13:18.182735 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.182706    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.187119 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.187085    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 26 18:13:18.187119 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.187109    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-q7bcj\""
Jul 26 18:13:18.187318 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.187160    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 26 18:13:18.187416 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.187377    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 26 18:13:18.187475 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.187429    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 26 18:13:18.187475 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.187379    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-9kmmtk08nkbas\""
Jul 26 18:13:18.195139 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.195109    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"]
Jul 26 18:13:18.248865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.248829    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d4c49e-1ab3-4773-9345-937697b92a9b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.249021 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.248878    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16d4c49e-1ab3-4773-9345-937697b92a9b-secret-metrics-server-tls\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.249021 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.248958    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lblkb\" (UniqueName: \"kubernetes.io/projected/16d4c49e-1ab3-4773-9345-937697b92a9b-kube-api-access-lblkb\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.249148 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.249058    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16d4c49e-1ab3-4773-9345-937697b92a9b-metrics-server-audit-profiles\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.249148 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.249092    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16d4c49e-1ab3-4773-9345-937697b92a9b-client-ca-bundle\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.249148 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.249116    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16d4c49e-1ab3-4773-9345-937697b92a9b-audit-log\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.249296 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.249156    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16d4c49e-1ab3-4773-9345-937697b92a9b-secret-metrics-server-client-certs\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.351410 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.350668    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16d4c49e-1ab3-4773-9345-937697b92a9b-secret-metrics-server-client-certs\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.351410 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.350754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d4c49e-1ab3-4773-9345-937697b92a9b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.351410 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.350782    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16d4c49e-1ab3-4773-9345-937697b92a9b-secret-metrics-server-tls\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.351410 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.350815    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lblkb\" (UniqueName: \"kubernetes.io/projected/16d4c49e-1ab3-4773-9345-937697b92a9b-kube-api-access-lblkb\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.351410 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.350892    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16d4c49e-1ab3-4773-9345-937697b92a9b-metrics-server-audit-profiles\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.351410 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.350919    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16d4c49e-1ab3-4773-9345-937697b92a9b-client-ca-bundle\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.351410 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.350943    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16d4c49e-1ab3-4773-9345-937697b92a9b-audit-log\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.351410 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.351308    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16d4c49e-1ab3-4773-9345-937697b92a9b-audit-log\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.351964 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.351657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d4c49e-1ab3-4773-9345-937697b92a9b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.352470 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.352195    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16d4c49e-1ab3-4773-9345-937697b92a9b-metrics-server-audit-profiles\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.355404 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.355359    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16d4c49e-1ab3-4773-9345-937697b92a9b-client-ca-bundle\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.355606 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.355572    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16d4c49e-1ab3-4773-9345-937697b92a9b-secret-metrics-server-tls\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.355783 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.355736    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16d4c49e-1ab3-4773-9345-937697b92a9b-secret-metrics-server-client-certs\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.360229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.360175    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lblkb\" (UniqueName: \"kubernetes.io/projected/16d4c49e-1ab3-4773-9345-937697b92a9b-kube-api-access-lblkb\") pod \"metrics-server-5c9bd8f97f-6d9n2\" (UID: \"16d4c49e-1ab3-4773-9345-937697b92a9b\") " pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.439864 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.439328    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ndlbj"
Jul 26 18:13:18.499056 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.498795    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:18.523187 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.523152    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sp82r" event={"ID":"7759bf11-07c1-4314-8133-74c3e58595fe","Type":"ContainerStarted","Data":"2947e8bc87c9a7c9f15f64d840ce3c15cace51cd8767c4b26aad48c91be7dbdf"}
Jul 26 18:13:18.527669 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.527467    2559 generic.go:356] "Generic (PLEG): container finished" podID="3dd18ce3-429f-49d0-8397-2a369659fe31" containerID="d6389ae4de64b4893bb1a7732e584a7fca0e7f9a0c9f015cb7ac3d4dbf400a35" exitCode=0
Jul 26 18:13:18.527669 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.527547    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dd18ce3-429f-49d0-8397-2a369659fe31","Type":"ContainerDied","Data":"d6389ae4de64b4893bb1a7732e584a7fca0e7f9a0c9f015cb7ac3d4dbf400a35"}
Jul 26 18:13:18.531991 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.531955    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jlkb4" event={"ID":"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4","Type":"ContainerStarted","Data":"39faac1f9ee1ed90924732fb51c90ddca271b1468f3f15531780891983dca48f"}
Jul 26 18:13:18.532129 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.532017    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-jlkb4" event={"ID":"618d2dc6-77cc-49e1-a892-b3f1f0a4eea4","Type":"ContainerStarted","Data":"150ef6108975f1ff285bc328bc068bffb07bad259e9a82878d9d40dde92c9b4c"}
Jul 26 18:13:18.535801 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.535771    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx" event={"ID":"4089ec57-7c50-4c25-9921-d575c935d3e2","Type":"ContainerStarted","Data":"a7c5d9300391976e1d4c594773808b182e3b08bfaf27c508cba706c09e7808de"}
Jul 26 18:13:18.548956 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.548395    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sp82r" podStartSLOduration=4.310906266 podStartE2EDuration="5.54837186s" totalImagesPullingTime="1.109128445s" totalInitContainerRuntime="128.337149ms" isStatefulPod=true podCreationTimestamp="2026-07-26 18:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:18.547326437 +0000 UTC m=+69.871848536" watchObservedRunningTime="2026-07-26 18:13:18.54837186 +0000 UTC m=+69.872893959"
Jul 26 18:13:18.586175 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.585917    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"]
Jul 26 18:13:18.591683 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.591053    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:18.594690 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.594661    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jrdjk\""
Jul 26 18:13:18.594989 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.594970    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 26 18:13:18.606710 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.606676    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"]
Jul 26 18:13:18.608477 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.608417    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-jlkb4" podStartSLOduration=67.834636573 podStartE2EDuration="1m9.608397038s" totalImagesPullingTime="1.773760465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:18.60103679 +0000 UTC m=+69.925558888" watchObservedRunningTime="2026-07-26 18:13:18.608397038 +0000 UTC m=+69.932919139"
Jul 26 18:13:18.655609 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.653438    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c45df73a-c600-4d13-9694-a5414a3e543e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-7zz4x\" (UID: \"c45df73a-c600-4d13-9694-a5414a3e543e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:18.655609 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.653799    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6bcp\" (UniqueName: \"kubernetes.io/projected/c45df73a-c600-4d13-9694-a5414a3e543e-kube-api-access-m6bcp\") pod \"monitoring-plugin-68b7fc9574-7zz4x\" (UID: \"c45df73a-c600-4d13-9694-a5414a3e543e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:18.712415 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.712290    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"]
Jul 26 18:13:18.755349 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.755308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c45df73a-c600-4d13-9694-a5414a3e543e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-7zz4x\" (UID: \"c45df73a-c600-4d13-9694-a5414a3e543e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:18.755526 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.755405    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6bcp\" (UniqueName: \"kubernetes.io/projected/c45df73a-c600-4d13-9694-a5414a3e543e-kube-api-access-m6bcp\") pod \"monitoring-plugin-68b7fc9574-7zz4x\" (UID: \"c45df73a-c600-4d13-9694-a5414a3e543e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:18.759222 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.759165    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c45df73a-c600-4d13-9694-a5414a3e543e-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-7zz4x\" (UID: \"c45df73a-c600-4d13-9694-a5414a3e543e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:18.765609 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.765549    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6bcp\" (UniqueName: \"kubernetes.io/projected/c45df73a-c600-4d13-9694-a5414a3e543e-kube-api-access-m6bcp\") pod \"monitoring-plugin-68b7fc9574-7zz4x\" (UID: \"c45df73a-c600-4d13-9694-a5414a3e543e\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:18.922266 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:18.921983    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:19.613251 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.613217    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dbcd8df67-mhf2g"]
Jul 26 18:13:19.616659 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.616633    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.630077 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.630023    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dbcd8df67-mhf2g"]
Jul 26 18:13:19.662601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.662557    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-oauth-serving-cert\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.662601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.662607    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2ww9\" (UniqueName: \"kubernetes.io/projected/a9ca9074-61eb-4515-88fe-2d4450d3aa34-kube-api-access-j2ww9\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.662865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.662814    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-config\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.662865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.662855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-trusted-ca-bundle\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.662960 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.662941    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-service-ca\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.663095 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.663053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-oauth-config\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.663233 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.663092    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-serving-cert\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.764423 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.764377    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-oauth-serving-cert\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.764423 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.764432    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2ww9\" (UniqueName: \"kubernetes.io/projected/a9ca9074-61eb-4515-88fe-2d4450d3aa34-kube-api-access-j2ww9\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.764731 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.764481    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-config\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.764731 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.764504    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-trusted-ca-bundle\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.764731 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.764552    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-service-ca\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.764731 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.764612    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-oauth-config\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.764731 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.764660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-serving-cert\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.765245 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.765219    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-oauth-serving-cert\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.765456 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.765431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-config\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.765600 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.765553    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-trusted-ca-bundle\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.765763 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.765738    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-service-ca\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.767335 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.767306    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-oauth-config\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.767476 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.767454    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-serving-cert\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.774328 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.774304    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2ww9\" (UniqueName: \"kubernetes.io/projected/a9ca9074-61eb-4515-88fe-2d4450d3aa34-kube-api-access-j2ww9\") pod \"console-dbcd8df67-mhf2g\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") " pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:19.927313 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:19.927262    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:20.222756 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.222667    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 18:13:20.228599 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.228566    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.234234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.233372    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3p1jgnpvl0mot\""
Jul 26 18:13:20.234234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.233561    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 26 18:13:20.234234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.233825    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 26 18:13:20.234234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.233827    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 26 18:13:20.234234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.233990    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 26 18:13:20.234234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.234055    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 26 18:13:20.234234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.234156    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 26 18:13:20.234234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.234217    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 26 18:13:20.234925 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.234902    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 26 18:13:20.235059 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.234934    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 26 18:13:20.235059 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.234903    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ks4pt\""
Jul 26 18:13:20.235502 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.235480    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 26 18:13:20.235771 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.235751    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 26 18:13:20.235870 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.235832    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 26 18:13:20.237304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.237281    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 26 18:13:20.241811 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.241788    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 26 18:13:20.255574 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.255536    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.274905    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.274953    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.274976    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275005    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275068    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfnxx\" (UniqueName: \"kubernetes.io/projected/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-kube-api-access-wfnxx\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275139    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275172    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275203    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275263    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-config\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275289    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-config-out\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275313    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275346    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.275738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.276590 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275407    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.276590 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.276590 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275454    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-web-config\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.276590 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.275476    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.325131 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:20.325092    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d4c49e_1ab3_4773_9345_937697b92a9b.slice/crio-d9a98764da440018d4adada6904833043b46e4b42d7e8911e16e25e19a87dcb9 WatchSource:0}: Error finding container d9a98764da440018d4adada6904833043b46e4b42d7e8911e16e25e19a87dcb9: Status 404 returned error can't find the container with id d9a98764da440018d4adada6904833043b46e4b42d7e8911e16e25e19a87dcb9
Jul 26 18:13:20.376556 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.376509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.376760 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.376576    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.376760 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.376638    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.376760 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.376673    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.376760 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.376695    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfnxx\" (UniqueName: \"kubernetes.io/projected/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-kube-api-access-wfnxx\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.376760 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.376736    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377041 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.376765    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377041 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.376805    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377218 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377309 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377244    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377309 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377280    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-config\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377310    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-config-out\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377318    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377335    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377400    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377585 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377426    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377585 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377472    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377585 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377585 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377531    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-web-config\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377585 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.377855 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.377610    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.378337 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.378003    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.378337 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.378037    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.378337 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.378091    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.378794 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.378686    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.383251 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.383224    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.383251 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.383239    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.383740 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.383712    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.383828 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.383773    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-config-out\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.383828 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.383806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.384480 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.384438    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-web-config\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.384642 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.384605    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.385523 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.385437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.385924 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.385715    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-config\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.386123 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.386081    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.386123 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.386087    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.386889 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.386604    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.387681 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.387637    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.388512 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.388475    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfnxx\" (UniqueName: \"kubernetes.io/projected/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-kube-api-access-wfnxx\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.389289 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.389264    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.541827 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.541701    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:20.553391 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:20.553355    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2" event={"ID":"16d4c49e-1ab3-4773-9345-937697b92a9b","Type":"ContainerStarted","Data":"d9a98764da440018d4adada6904833043b46e4b42d7e8911e16e25e19a87dcb9"}
Jul 26 18:13:22.119226 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.119188    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dbcd8df67-mhf2g"]
Jul 26 18:13:22.374760 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:22.374726    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ca9074_61eb_4515_88fe_2d4450d3aa34.slice/crio-3edda9de06496aed95ef89ed974fd5b53ab71e6063ba16938715c7820aed99da WatchSource:0}: Error finding container 3edda9de06496aed95ef89ed974fd5b53ab71e6063ba16938715c7820aed99da: Status 404 returned error can't find the container with id 3edda9de06496aed95ef89ed974fd5b53ab71e6063ba16938715c7820aed99da
Jul 26 18:13:22.560084 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.559659    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"]
Jul 26 18:13:22.562232 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:22.562194    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45df73a_c600_4d13_9694_a5414a3e543e.slice/crio-b2697541bd1dbbebd22fa4da3edbd3347eb4f769e9fbfe46d04cf36daee6634c WatchSource:0}: Error finding container b2697541bd1dbbebd22fa4da3edbd3347eb4f769e9fbfe46d04cf36daee6634c: Status 404 returned error can't find the container with id b2697541bd1dbbebd22fa4da3edbd3347eb4f769e9fbfe46d04cf36daee6634c
Jul 26 18:13:22.566630 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.566581    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pmgnx" event={"ID":"8dc6d460-fedb-4a07-89c0-3a16a76ed30b","Type":"ContainerStarted","Data":"6dc0f393a6aa1562364873a48803be94ff64afab93e934411922b231aaa7e8f2"}
Jul 26 18:13:22.566733 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.566687    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:13:22.573318 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.571866    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbcd8df67-mhf2g" event={"ID":"a9ca9074-61eb-4515-88fe-2d4450d3aa34","Type":"ContainerStarted","Data":"68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5"}
Jul 26 18:13:22.573318 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.571920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbcd8df67-mhf2g" event={"ID":"a9ca9074-61eb-4515-88fe-2d4450d3aa34","Type":"ContainerStarted","Data":"3edda9de06496aed95ef89ed974fd5b53ab71e6063ba16938715c7820aed99da"}
Jul 26 18:13:22.576992 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.576887    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx" event={"ID":"4089ec57-7c50-4c25-9921-d575c935d3e2","Type":"ContainerStarted","Data":"2c9b75c5c1907b0dbf5f26f6f1cb251f4f0e6ba87bd56f7d8289dad468ba8547"}
Jul 26 18:13:22.586830 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.585752    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pmgnx" podStartSLOduration=67.04433193 podStartE2EDuration="1m13.585731709s" totalImagesPullingTime="6.541399779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:22.585053763 +0000 UTC m=+73.909575864" watchObservedRunningTime="2026-07-26 18:13:22.585731709 +0000 UTC m=+73.910253808"
Jul 26 18:13:22.598653 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.598546    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 26 18:13:22.609027 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:22.608808    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dbcd8df67-mhf2g" podStartSLOduration=3.6087849690000002 podStartE2EDuration="3.608784969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:22.607362442 +0000 UTC m=+73.931884539" watchObservedRunningTime="2026-07-26 18:13:22.608784969 +0000 UTC m=+73.933307069"
Jul 26 18:13:22.609246 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:13:22.609065    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280e3f0a_585c_4a7f_9ce1_0e1c4f0bd78b.slice/crio-8cd38f1f1715ac8ed12e725f7838085c8efa7e17ef3769da4fd03e7841cb873c WatchSource:0}: Error finding container 8cd38f1f1715ac8ed12e725f7838085c8efa7e17ef3769da4fd03e7841cb873c: Status 404 returned error can't find the container with id 8cd38f1f1715ac8ed12e725f7838085c8efa7e17ef3769da4fd03e7841cb873c
Jul 26 18:13:23.587260 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.587138    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx" event={"ID":"4089ec57-7c50-4c25-9921-d575c935d3e2","Type":"ContainerStarted","Data":"57158720acbee35af85c942f20ed8ae109470df1c18ebda784d57b4c39b3565b"}
Jul 26 18:13:23.587260 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.587228    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx" event={"ID":"4089ec57-7c50-4c25-9921-d575c935d3e2","Type":"ContainerStarted","Data":"9501977f5a46eeec6d306ef84c7c5b48c060ccc22f589f5ef48a00a37c597e03"}
Jul 26 18:13:23.589693 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.589658    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2" event={"ID":"16d4c49e-1ab3-4773-9345-937697b92a9b","Type":"ContainerStarted","Data":"04d50291864ab59c9e4fccebb2f2d97c96f8d008952b2428197f2384c72abb8a"}
Jul 26 18:13:23.591256 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.591228    2559 generic.go:356] "Generic (PLEG): container finished" podID="280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b" containerID="22146ac8e0b88c5dd72984299202890b98c986a1c832b2929947ab20b808451c" exitCode=0
Jul 26 18:13:23.591348 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.591307    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b","Type":"ContainerDied","Data":"22146ac8e0b88c5dd72984299202890b98c986a1c832b2929947ab20b808451c"}
Jul 26 18:13:23.591400 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.591384    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b","Type":"ContainerStarted","Data":"8cd38f1f1715ac8ed12e725f7838085c8efa7e17ef3769da4fd03e7841cb873c"}
Jul 26 18:13:23.592601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.592570    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x" event={"ID":"c45df73a-c600-4d13-9694-a5414a3e543e","Type":"ContainerStarted","Data":"b2697541bd1dbbebd22fa4da3edbd3347eb4f769e9fbfe46d04cf36daee6634c"}
Jul 26 18:13:23.594159 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.594131    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n4ng2" event={"ID":"b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b","Type":"ContainerStarted","Data":"b96d64988f60f61247f09ec1c13977481de11e8fbb66d8a7c9c1a5e70c69e504"}
Jul 26 18:13:23.597288 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.597266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dd18ce3-429f-49d0-8397-2a369659fe31","Type":"ContainerStarted","Data":"8f1c45f3e94fae50f3d24ded4b63c45dcc3d5670491d3d91f3bdc6262f936cc9"}
Jul 26 18:13:23.597356 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.597335    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dd18ce3-429f-49d0-8397-2a369659fe31","Type":"ContainerStarted","Data":"23f976c2607f44cfe479bbab6ff4cbb121187156b1bb65761dc27fd8c873bb4a"}
Jul 26 18:13:23.597409 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.597376    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dd18ce3-429f-49d0-8397-2a369659fe31","Type":"ContainerStarted","Data":"06e3d82a10c7b6afefd3bf6f68242ee9d26355dcb6584680c770277458d169e1"}
Jul 26 18:13:23.597469 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.597416    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dd18ce3-429f-49d0-8397-2a369659fe31","Type":"ContainerStarted","Data":"79a09a2e0fb4d94a5b79739bd0bbc227d50e6d8555af836f8688ee7320548021"}
Jul 26 18:13:23.597469 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.597457    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dd18ce3-429f-49d0-8397-2a369659fe31","Type":"ContainerStarted","Data":"7d670e58ec9f6dab4185817027a0a61bf7c97691fbc545bac33ca68df34f85bd"}
Jul 26 18:13:23.609526 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.609455    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2" podStartSLOduration=2.748753616 podStartE2EDuration="5.609435437s" totalImagesPullingTime="2.860681821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:23.60856451 +0000 UTC m=+74.933086608" watchObservedRunningTime="2026-07-26 18:13:23.609435437 +0000 UTC m=+74.933957535"
Jul 26 18:13:23.623454 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:23.623397    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n4ng2" podStartSLOduration=67.820285578 podStartE2EDuration="1m14.623380028s" totalImagesPullingTime="6.80309445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 18:12:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:23.623071933 +0000 UTC m=+74.947594032" watchObservedRunningTime="2026-07-26 18:13:23.623380028 +0000 UTC m=+74.947902119"
Jul 26 18:13:24.606041 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.605950    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx" event={"ID":"4089ec57-7c50-4c25-9921-d575c935d3e2","Type":"ContainerStarted","Data":"327d75a6577371c1ab1f1099013ad86e111f35044ae34c6bd2293f335fd0927a"}
Jul 26 18:13:24.606485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.606041    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx" event={"ID":"4089ec57-7c50-4c25-9921-d575c935d3e2","Type":"ContainerStarted","Data":"4b59a03676f0c07581a585f16fc537482913ceaf60d79c524dd9250f89abe004"}
Jul 26 18:13:24.606485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.606088    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx" event={"ID":"4089ec57-7c50-4c25-9921-d575c935d3e2","Type":"ContainerStarted","Data":"fb7584ab06222a1c71ad8971c0c24551d914851cc2eb564da4ea8c1418e9f7cb"}
Jul 26 18:13:24.606485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.606184    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:24.607802 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.607773    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x" event={"ID":"c45df73a-c600-4d13-9694-a5414a3e543e","Type":"ContainerStarted","Data":"33938e8c9f2dc02cb6fdecaa7cdd45c6c28cf3a504d1ce5ac32641de407fd99d"}
Jul 26 18:13:24.608027 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.607923    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:24.612599 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.612515    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3dd18ce3-429f-49d0-8397-2a369659fe31","Type":"ContainerStarted","Data":"8f27054d84778c4efc447c8f764a7b66c4dc8075c1bf9ae047c0ded66820ed10"}
Jul 26 18:13:24.613497 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.613476    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x"
Jul 26 18:13:24.635316 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.635250    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx" podStartSLOduration=3.912431847 podStartE2EDuration="9.635226406s" totalImagesPullingTime="5.722794559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:13:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:24.632273528 +0000 UTC m=+75.956795649" watchObservedRunningTime="2026-07-26 18:13:24.635226406 +0000 UTC m=+75.959748709"
Jul 26 18:13:24.665686 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.665602    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.27112572 podStartE2EDuration="10.665584186s" totalImagesPullingTime="6.373028089s" totalInitContainerRuntime="21.430377ms" isStatefulPod=false podCreationTimestamp="2026-07-26 18:13:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:24.664994115 +0000 UTC m=+75.989516215" watchObservedRunningTime="2026-07-26 18:13:24.665584186 +0000 UTC m=+75.990106282"
Jul 26 18:13:24.692405 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:24.692240    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-7zz4x" podStartSLOduration=5.038272766 podStartE2EDuration="6.692215241s" totalImagesPullingTime="1.653942475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:24.69073924 +0000 UTC m=+76.015261341" watchObservedRunningTime="2026-07-26 18:13:24.692215241 +0000 UTC m=+76.016737341"
Jul 26 18:13:27.631034 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:27.630945    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b","Type":"ContainerStarted","Data":"833ca8c116117c65b74a1b029e0649053e8527e7aff051aefbb4419c3d73b866"}
Jul 26 18:13:27.631430 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:27.631050    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b","Type":"ContainerStarted","Data":"03e1bae8718b53f6ecd24fe9d9cc3330a28214f4e222c4ba71b819cc20e96ad5"}
Jul 26 18:13:27.631430 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:27.631076    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b","Type":"ContainerStarted","Data":"a7c933cd95ac2e76f36004ebcc71ab8a9e5aaf731b1aa1bdc137c69ed098eeb3"}
Jul 26 18:13:27.631430 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:27.631100    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b","Type":"ContainerStarted","Data":"819e09512f33e77fd52dacc470ff1eb3ce4a342b4bd3bc76c6edda09eaad531b"}
Jul 26 18:13:27.631430 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:27.631126    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b","Type":"ContainerStarted","Data":"1354ba0d0a59d54253a319390b263a8f8af40ad74a37d41a36096891674402f4"}
Jul 26 18:13:27.631430 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:27.631149    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b","Type":"ContainerStarted","Data":"16272c8d9a6d12e5b7fb664684de55581fd4d3223889ef4d240219ef2d24c6c2"}
Jul 26 18:13:27.662543 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:27.662488    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.207364445 podStartE2EDuration="7.662467704s" totalImagesPullingTime="3.423767653s" totalInitContainerRuntime="31.335606ms" isStatefulPod=false podCreationTimestamp="2026-07-26 18:13:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:13:27.660043031 +0000 UTC m=+78.984565122" watchObservedRunningTime="2026-07-26 18:13:27.662467704 +0000 UTC m=+78.986989804"
Jul 26 18:13:29.927761 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:29.927723    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:29.928168 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:29.927815    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:29.932746 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:29.932722    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:30.542288 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:30.542248    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:13:30.624283 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:30.624177    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8496b7b7c8-9dxhx"
Jul 26 18:13:30.647588 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:30.647560    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:13:30.704370 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:30.704340    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5bdb77c7-5rp4n"]
Jul 26 18:13:38.498911 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:38.498873    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:38.499422 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:38.498983    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:53.600601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:53.600570    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pmgnx"
Jul 26 18:13:55.725447 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:55.725356    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c5bdb77c7-5rp4n" podUID="4cb916a1-34e5-491c-851d-51c0606f3f20" containerName="console" containerID="cri-o://ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556" gracePeriod=15
Jul 26 18:13:56.005877 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.005851    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:56.113455 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.113421    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-oauth-config\") pod \"4cb916a1-34e5-491c-851d-51c0606f3f20\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") "
Jul 26 18:13:56.113656 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.113472    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-service-ca\") pod \"4cb916a1-34e5-491c-851d-51c0606f3f20\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") "
Jul 26 18:13:56.113656 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.113554    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-console-config\") pod \"4cb916a1-34e5-491c-851d-51c0606f3f20\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") "
Jul 26 18:13:56.113656 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.113581    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9wtp\" (UniqueName: \"kubernetes.io/projected/4cb916a1-34e5-491c-851d-51c0606f3f20-kube-api-access-g9wtp\") pod \"4cb916a1-34e5-491c-851d-51c0606f3f20\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") "
Jul 26 18:13:56.113656 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.113633    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-trusted-ca-bundle\") pod \"4cb916a1-34e5-491c-851d-51c0606f3f20\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") "
Jul 26 18:13:56.113876 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.113672    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-serving-cert\") pod \"4cb916a1-34e5-491c-851d-51c0606f3f20\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") "
Jul 26 18:13:56.113876 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.113751    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-oauth-serving-cert\") pod \"4cb916a1-34e5-491c-851d-51c0606f3f20\" (UID: \"4cb916a1-34e5-491c-851d-51c0606f3f20\") "
Jul 26 18:13:56.113981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.113860    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-service-ca" (OuterVolumeSpecName: "service-ca") pod "4cb916a1-34e5-491c-851d-51c0606f3f20" (UID: "4cb916a1-34e5-491c-851d-51c0606f3f20"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:13:56.113981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.113944    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-console-config" (OuterVolumeSpecName: "console-config") pod "4cb916a1-34e5-491c-851d-51c0606f3f20" (UID: "4cb916a1-34e5-491c-851d-51c0606f3f20"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:13:56.114099 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.114080    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-console-config\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:13:56.114099 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.114098    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-service-ca\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:13:56.114202 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.114163    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4cb916a1-34e5-491c-851d-51c0606f3f20" (UID: "4cb916a1-34e5-491c-851d-51c0606f3f20"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:13:56.114256 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.114196    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4cb916a1-34e5-491c-851d-51c0606f3f20" (UID: "4cb916a1-34e5-491c-851d-51c0606f3f20"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:13:56.116124 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.116092    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb916a1-34e5-491c-851d-51c0606f3f20-kube-api-access-g9wtp" (OuterVolumeSpecName: "kube-api-access-g9wtp") pod "4cb916a1-34e5-491c-851d-51c0606f3f20" (UID: "4cb916a1-34e5-491c-851d-51c0606f3f20"). InnerVolumeSpecName "kube-api-access-g9wtp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:13:56.116317 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.116300    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4cb916a1-34e5-491c-851d-51c0606f3f20" (UID: "4cb916a1-34e5-491c-851d-51c0606f3f20"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:13:56.116454 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.116427    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4cb916a1-34e5-491c-851d-51c0606f3f20" (UID: "4cb916a1-34e5-491c-851d-51c0606f3f20"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:13:56.215305 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.215270    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-oauth-serving-cert\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:13:56.215305 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.215299    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-oauth-config\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:13:56.215305 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.215309    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9wtp\" (UniqueName: \"kubernetes.io/projected/4cb916a1-34e5-491c-851d-51c0606f3f20-kube-api-access-g9wtp\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:13:56.215305 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.215318    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4cb916a1-34e5-491c-851d-51c0606f3f20-trusted-ca-bundle\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:13:56.215305 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.215328    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4cb916a1-34e5-491c-851d-51c0606f3f20-console-serving-cert\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:13:56.754077 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.754045    2559 generic.go:356] "Generic (PLEG): container finished" podID="4cb916a1-34e5-491c-851d-51c0606f3f20" containerID="ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556" exitCode=0
Jul 26 18:13:56.754565 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.754101    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5bdb77c7-5rp4n"
Jul 26 18:13:56.754565 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.754108    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5bdb77c7-5rp4n" event={"ID":"4cb916a1-34e5-491c-851d-51c0606f3f20","Type":"ContainerDied","Data":"ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556"}
Jul 26 18:13:56.754565 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.754212    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5bdb77c7-5rp4n" event={"ID":"4cb916a1-34e5-491c-851d-51c0606f3f20","Type":"ContainerDied","Data":"9bc8c28b8cb72abaaf93a7a6dd7abab1eea5e811bf6d3cf00a0b4be78fcc2a73"}
Jul 26 18:13:56.754565 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.754232    2559 scope.go:117] "RemoveContainer" containerID="ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556"
Jul 26 18:13:56.763804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.763749    2559 scope.go:117] "RemoveContainer" containerID="ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556"
Jul 26 18:13:56.764047 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:13:56.764028    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556\": container with ID starting with ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556 not found: ID does not exist" containerID="ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556"
Jul 26 18:13:56.764112 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.764059    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556"} err="failed to get container status \"ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556\": rpc error: code = NotFound desc = could not find container \"ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556\": container with ID starting with ee1c76027410a1472e07c71e0b435516605dd78e87fc564387cca28936cc2556 not found: ID does not exist"
Jul 26 18:13:56.778853 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.777596    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5bdb77c7-5rp4n"]
Jul 26 18:13:56.782246 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:56.782206    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5bdb77c7-5rp4n"]
Jul 26 18:13:57.176014 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:57.175979    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cb916a1-34e5-491c-851d-51c0606f3f20" path="/var/lib/kubelet/pods/4cb916a1-34e5-491c-851d-51c0606f3f20/volumes"
Jul 26 18:13:58.504695 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:58.504665    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:13:58.508965 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:13:58.508936    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c9bd8f97f-6d9n2"
Jul 26 18:14:20.542480 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:14:20.542431    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:14:20.562255 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:14:20.562226    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:14:20.864934 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:14:20.864860    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 26 18:15:00.801000 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:00.800918    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dbcd8df67-mhf2g"]
Jul 26 18:15:25.820565 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:25.820519    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-dbcd8df67-mhf2g" podUID="a9ca9074-61eb-4515-88fe-2d4450d3aa34" containerName="console" containerID="cri-o://68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5" gracePeriod=15
Jul 26 18:15:26.065709 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.065684    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:15:26.111546 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.111445    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-oauth-serving-cert\") pod \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") "
Jul 26 18:15:26.111546 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.111490    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-serving-cert\") pod \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") "
Jul 26 18:15:26.111825 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.111547    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-trusted-ca-bundle\") pod \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") "
Jul 26 18:15:26.111825 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.111585    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-oauth-config\") pod \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") "
Jul 26 18:15:26.111825 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.111640    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-service-ca\") pod \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") "
Jul 26 18:15:26.111825 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.111684    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-config\") pod \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") "
Jul 26 18:15:26.111825 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.111714    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2ww9\" (UniqueName: \"kubernetes.io/projected/a9ca9074-61eb-4515-88fe-2d4450d3aa34-kube-api-access-j2ww9\") pod \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\" (UID: \"a9ca9074-61eb-4515-88fe-2d4450d3aa34\") "
Jul 26 18:15:26.112078 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.111897    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a9ca9074-61eb-4515-88fe-2d4450d3aa34" (UID: "a9ca9074-61eb-4515-88fe-2d4450d3aa34"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:15:26.112078 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.112000    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-oauth-serving-cert\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:15:26.112359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.112325    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-service-ca" (OuterVolumeSpecName: "service-ca") pod "a9ca9074-61eb-4515-88fe-2d4450d3aa34" (UID: "a9ca9074-61eb-4515-88fe-2d4450d3aa34"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:15:26.112462 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.112362    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a9ca9074-61eb-4515-88fe-2d4450d3aa34" (UID: "a9ca9074-61eb-4515-88fe-2d4450d3aa34"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:15:26.112826 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.112760    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-config" (OuterVolumeSpecName: "console-config") pod "a9ca9074-61eb-4515-88fe-2d4450d3aa34" (UID: "a9ca9074-61eb-4515-88fe-2d4450d3aa34"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:15:26.112987 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.112961    2559 generic.go:356] "Generic (PLEG): container finished" podID="a9ca9074-61eb-4515-88fe-2d4450d3aa34" containerID="68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5" exitCode=0
Jul 26 18:15:26.113072 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.113027    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dbcd8df67-mhf2g"
Jul 26 18:15:26.113072 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.113046    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbcd8df67-mhf2g" event={"ID":"a9ca9074-61eb-4515-88fe-2d4450d3aa34","Type":"ContainerDied","Data":"68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5"}
Jul 26 18:15:26.113169 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.113132    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dbcd8df67-mhf2g" event={"ID":"a9ca9074-61eb-4515-88fe-2d4450d3aa34","Type":"ContainerDied","Data":"3edda9de06496aed95ef89ed974fd5b53ab71e6063ba16938715c7820aed99da"}
Jul 26 18:15:26.113169 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.113165    2559 scope.go:117] "RemoveContainer" containerID="68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5"
Jul 26 18:15:26.114452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.114424    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a9ca9074-61eb-4515-88fe-2d4450d3aa34" (UID: "a9ca9074-61eb-4515-88fe-2d4450d3aa34"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:15:26.114582 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.114521    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a9ca9074-61eb-4515-88fe-2d4450d3aa34" (UID: "a9ca9074-61eb-4515-88fe-2d4450d3aa34"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:15:26.114761 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.114736    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9ca9074-61eb-4515-88fe-2d4450d3aa34-kube-api-access-j2ww9" (OuterVolumeSpecName: "kube-api-access-j2ww9") pod "a9ca9074-61eb-4515-88fe-2d4450d3aa34" (UID: "a9ca9074-61eb-4515-88fe-2d4450d3aa34"). InnerVolumeSpecName "kube-api-access-j2ww9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:15:26.129341 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.129317    2559 scope.go:117] "RemoveContainer" containerID="68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5"
Jul 26 18:15:26.129675 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:15:26.129655    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5\": container with ID starting with 68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5 not found: ID does not exist" containerID="68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5"
Jul 26 18:15:26.129733 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.129686    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5"} err="failed to get container status \"68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5\": rpc error: code = NotFound desc = could not find container \"68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5\": container with ID starting with 68c6ada19cd6655ca67ce0aa57b2b954c30d0163b39b0b4a47ff2880bb83aea5 not found: ID does not exist"
Jul 26 18:15:26.212798 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.212758    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-config\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:15:26.212798 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.212790    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2ww9\" (UniqueName: \"kubernetes.io/projected/a9ca9074-61eb-4515-88fe-2d4450d3aa34-kube-api-access-j2ww9\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:15:26.212798 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.212805    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-serving-cert\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:15:26.213034 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.212814    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-trusted-ca-bundle\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:15:26.213034 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.212823    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9ca9074-61eb-4515-88fe-2d4450d3aa34-console-oauth-config\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:15:26.213034 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.212831    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9ca9074-61eb-4515-88fe-2d4450d3aa34-service-ca\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:15:26.437198 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.437152    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dbcd8df67-mhf2g"]
Jul 26 18:15:26.440808 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:26.440779    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dbcd8df67-mhf2g"]
Jul 26 18:15:27.176219 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:15:27.176184    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9ca9074-61eb-4515-88fe-2d4450d3aa34" path="/var/lib/kubelet/pods/a9ca9074-61eb-4515-88fe-2d4450d3aa34/volumes"
Jul 26 18:16:32.723209 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.723116    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8qntt"]
Jul 26 18:16:32.723686 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.723496    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cb916a1-34e5-491c-851d-51c0606f3f20" containerName="console"
Jul 26 18:16:32.723686 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.723511    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cb916a1-34e5-491c-851d-51c0606f3f20" containerName="console"
Jul 26 18:16:32.723686 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.723525    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9ca9074-61eb-4515-88fe-2d4450d3aa34" containerName="console"
Jul 26 18:16:32.723686 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.723533    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9ca9074-61eb-4515-88fe-2d4450d3aa34" containerName="console"
Jul 26 18:16:32.723686 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.723585    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cb916a1-34e5-491c-851d-51c0606f3f20" containerName="console"
Jul 26 18:16:32.723686 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.723596    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9ca9074-61eb-4515-88fe-2d4450d3aa34" containerName="console"
Jul 26 18:16:32.726380 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.726366    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:32.732051 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.732023    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 26 18:16:32.732203 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.732138    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-c7mr7\""
Jul 26 18:16:32.733363 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.733343    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 26 18:16:32.741413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.741382    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8qntt"]
Jul 26 18:16:32.888757 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.888713    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5128b28d-0dfd-48d0-947d-139ec18cd32c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8qntt\" (UID: \"5128b28d-0dfd-48d0-947d-139ec18cd32c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:32.888939 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.888786    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xh64\" (UniqueName: \"kubernetes.io/projected/5128b28d-0dfd-48d0-947d-139ec18cd32c-kube-api-access-4xh64\") pod \"cert-manager-webhook-6486ffd997-8qntt\" (UID: \"5128b28d-0dfd-48d0-947d-139ec18cd32c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:32.990022 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.989934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5128b28d-0dfd-48d0-947d-139ec18cd32c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8qntt\" (UID: \"5128b28d-0dfd-48d0-947d-139ec18cd32c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:32.990022 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:32.990011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xh64\" (UniqueName: \"kubernetes.io/projected/5128b28d-0dfd-48d0-947d-139ec18cd32c-kube-api-access-4xh64\") pod \"cert-manager-webhook-6486ffd997-8qntt\" (UID: \"5128b28d-0dfd-48d0-947d-139ec18cd32c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:33.002353 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:33.002318    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5128b28d-0dfd-48d0-947d-139ec18cd32c-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8qntt\" (UID: \"5128b28d-0dfd-48d0-947d-139ec18cd32c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:33.003675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:33.003657    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xh64\" (UniqueName: \"kubernetes.io/projected/5128b28d-0dfd-48d0-947d-139ec18cd32c-kube-api-access-4xh64\") pod \"cert-manager-webhook-6486ffd997-8qntt\" (UID: \"5128b28d-0dfd-48d0-947d-139ec18cd32c\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:33.049486 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:33.049425    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:33.194150 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:33.194119    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8qntt"]
Jul 26 18:16:33.197302 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:16:33.197267    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5128b28d_0dfd_48d0_947d_139ec18cd32c.slice/crio-b6cb69866d89ee9ba8ea497cd2b96086724daa44a468592d513e8cb65283cdca WatchSource:0}: Error finding container b6cb69866d89ee9ba8ea497cd2b96086724daa44a468592d513e8cb65283cdca: Status 404 returned error can't find the container with id b6cb69866d89ee9ba8ea497cd2b96086724daa44a468592d513e8cb65283cdca
Jul 26 18:16:33.371472 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:33.371378    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt" event={"ID":"5128b28d-0dfd-48d0-947d-139ec18cd32c","Type":"ContainerStarted","Data":"b6cb69866d89ee9ba8ea497cd2b96086724daa44a468592d513e8cb65283cdca"}
Jul 26 18:16:37.390572 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:37.390531    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt" event={"ID":"5128b28d-0dfd-48d0-947d-139ec18cd32c","Type":"ContainerStarted","Data":"63343fab40af12c84f00c516197ff838cafa80ace9b4b14bf88bc8cb14d437bc"}
Jul 26 18:16:37.391056 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:37.390674    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:37.409455 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:37.409400    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt" podStartSLOduration=2.187430252 podStartE2EDuration="5.409382664s" totalImagesPullingTime="3.221952412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:16:37.408413769 +0000 UTC m=+268.732935867" watchObservedRunningTime="2026-07-26 18:16:37.409382664 +0000 UTC m=+268.733904761"
Jul 26 18:16:43.396589 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:43.396556    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8qntt"
Jul 26 18:16:46.555498 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.555443    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"]
Jul 26 18:16:46.562094 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.562060    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"
Jul 26 18:16:46.565258 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.565178    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 26 18:16:46.565258 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.565192    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 26 18:16:46.565480 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.565328    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-h9dfn\""
Jul 26 18:16:46.569185 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.569132    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"]
Jul 26 18:16:46.604452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.604377    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw6l4\" (UniqueName: \"kubernetes.io/projected/38d54870-7857-45b9-879a-7e9c81a82a56-kube-api-access-tw6l4\") pod \"openshift-lws-operator-fd8ccff4c-pdkf5\" (UID: \"38d54870-7857-45b9-879a-7e9c81a82a56\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"
Jul 26 18:16:46.604647 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.604487    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38d54870-7857-45b9-879a-7e9c81a82a56-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pdkf5\" (UID: \"38d54870-7857-45b9-879a-7e9c81a82a56\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"
Jul 26 18:16:46.704914 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.704870    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw6l4\" (UniqueName: \"kubernetes.io/projected/38d54870-7857-45b9-879a-7e9c81a82a56-kube-api-access-tw6l4\") pod \"openshift-lws-operator-fd8ccff4c-pdkf5\" (UID: \"38d54870-7857-45b9-879a-7e9c81a82a56\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"
Jul 26 18:16:46.704914 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.704920    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38d54870-7857-45b9-879a-7e9c81a82a56-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pdkf5\" (UID: \"38d54870-7857-45b9-879a-7e9c81a82a56\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"
Jul 26 18:16:46.705350 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.705328    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38d54870-7857-45b9-879a-7e9c81a82a56-tmp\") pod \"openshift-lws-operator-fd8ccff4c-pdkf5\" (UID: \"38d54870-7857-45b9-879a-7e9c81a82a56\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"
Jul 26 18:16:46.714849 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.714813    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw6l4\" (UniqueName: \"kubernetes.io/projected/38d54870-7857-45b9-879a-7e9c81a82a56-kube-api-access-tw6l4\") pod \"openshift-lws-operator-fd8ccff4c-pdkf5\" (UID: \"38d54870-7857-45b9-879a-7e9c81a82a56\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"
Jul 26 18:16:46.871565 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:46.871457    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"
Jul 26 18:16:47.004769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:47.004736    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5"]
Jul 26 18:16:47.007768 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:16:47.007738    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38d54870_7857_45b9_879a_7e9c81a82a56.slice/crio-c4fc039db16ab8fe12371fc0076f6ed111c71d30f5364a135b96f6d8803c8dd5 WatchSource:0}: Error finding container c4fc039db16ab8fe12371fc0076f6ed111c71d30f5364a135b96f6d8803c8dd5: Status 404 returned error can't find the container with id c4fc039db16ab8fe12371fc0076f6ed111c71d30f5364a135b96f6d8803c8dd5
Jul 26 18:16:47.431026 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:47.430989    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5" event={"ID":"38d54870-7857-45b9-879a-7e9c81a82a56","Type":"ContainerStarted","Data":"c4fc039db16ab8fe12371fc0076f6ed111c71d30f5364a135b96f6d8803c8dd5"}
Jul 26 18:16:50.446383 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:50.446346    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5" event={"ID":"38d54870-7857-45b9-879a-7e9c81a82a56","Type":"ContainerStarted","Data":"fd68c41cde80f53c355d24dd1a42a751a7c47f5e58100b5f065fed89bb2bea88"}
Jul 26 18:16:50.463906 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:16:50.463833    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-pdkf5" podStartSLOduration=1.793492131 podStartE2EDuration="4.463815641s" totalImagesPullingTime="2.67032351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:16:50.46308566 +0000 UTC m=+281.787607759" watchObservedRunningTime="2026-07-26 18:16:50.463815641 +0000 UTC m=+281.788337738"
Jul 26 18:17:06.230913 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.230876    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"]
Jul 26 18:17:06.234380 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.234363    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.237743 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.237716    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 26 18:17:06.238244 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.238227    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 26 18:17:06.238554 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.238533    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 26 18:17:06.238725 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.238702    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zw2hr\""
Jul 26 18:17:06.239022 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.239007    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 26 18:17:06.257998 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.257958    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/05604bb7-666e-4786-a350-bea356d5e5a5-manifests\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.258167 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.258149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/05604bb7-666e-4786-a350-bea356d5e5a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.258230 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.258193    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/05604bb7-666e-4786-a350-bea356d5e5a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.258286 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.258233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wnnn\" (UniqueName: \"kubernetes.io/projected/05604bb7-666e-4786-a350-bea356d5e5a5-kube-api-access-8wnnn\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.258420 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.258387    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05604bb7-666e-4786-a350-bea356d5e5a5-tmp\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.272294 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.272267    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"]
Jul 26 18:17:06.359249 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.359214    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05604bb7-666e-4786-a350-bea356d5e5a5-tmp\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.359459 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.359302    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/05604bb7-666e-4786-a350-bea356d5e5a5-manifests\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.359459 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.359346    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/05604bb7-666e-4786-a350-bea356d5e5a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.359459 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.359372    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/05604bb7-666e-4786-a350-bea356d5e5a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.359459 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.359396    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wnnn\" (UniqueName: \"kubernetes.io/projected/05604bb7-666e-4786-a350-bea356d5e5a5-kube-api-access-8wnnn\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.359804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.359774    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/05604bb7-666e-4786-a350-bea356d5e5a5-manifests\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.361691 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.361661    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/05604bb7-666e-4786-a350-bea356d5e5a5-tmp\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.361802 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.361778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/05604bb7-666e-4786-a350-bea356d5e5a5-webhook-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.361864 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.361839    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/05604bb7-666e-4786-a350-bea356d5e5a5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.372849 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.372825    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wnnn\" (UniqueName: \"kubernetes.io/projected/05604bb7-666e-4786-a350-bea356d5e5a5-kube-api-access-8wnnn\") pod \"opendatahub-operator-controller-manager-758747dbfc-mqvph\" (UID: \"05604bb7-666e-4786-a350-bea356d5e5a5\") " pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.545090 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.545006    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:06.683536 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:06.683514    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"]
Jul 26 18:17:06.685916 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:17:06.685882    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05604bb7_666e_4786_a350_bea356d5e5a5.slice/crio-7c35ae41085ed76fd65eda563af2e68c4348b4323bd6c96a99624f225927ee74 WatchSource:0}: Error finding container 7c35ae41085ed76fd65eda563af2e68c4348b4323bd6c96a99624f225927ee74: Status 404 returned error can't find the container with id 7c35ae41085ed76fd65eda563af2e68c4348b4323bd6c96a99624f225927ee74
Jul 26 18:17:07.518923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:07.518882    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph" event={"ID":"05604bb7-666e-4786-a350-bea356d5e5a5","Type":"ContainerStarted","Data":"7c35ae41085ed76fd65eda563af2e68c4348b4323bd6c96a99624f225927ee74"}
Jul 26 18:17:09.717822 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:09.717786    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:17:09.720155 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:09.717851    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:17:09.723229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:09.723209    2559 kubelet.go:1648] "Image garbage collection succeeded"
Jul 26 18:17:10.534836 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:10.534804    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph" event={"ID":"05604bb7-666e-4786-a350-bea356d5e5a5","Type":"ContainerStarted","Data":"aa48d810af2ca2a99002d1dc8c9b5e44caac5924bdbd26aa619996caa1aa5112"}
Jul 26 18:17:11.540558 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:11.540525    2559 generic.go:356] "Generic (PLEG): container finished" podID="05604bb7-666e-4786-a350-bea356d5e5a5" containerID="aa48d810af2ca2a99002d1dc8c9b5e44caac5924bdbd26aa619996caa1aa5112" exitCode=0
Jul 26 18:17:11.540979 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:11.540594    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph" event={"ID":"05604bb7-666e-4786-a350-bea356d5e5a5","Type":"ContainerDied","Data":"aa48d810af2ca2a99002d1dc8c9b5e44caac5924bdbd26aa619996caa1aa5112"}
Jul 26 18:17:11.541167 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:11.541150    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:17:12.553252 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:12.553217    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph" event={"ID":"05604bb7-666e-4786-a350-bea356d5e5a5","Type":"ContainerStarted","Data":"b67e5d96f4b729995b85f2f5942e0cdb4aa64e466fb9fd18d3fbb19b9ee27154"}
Jul 26 18:17:12.553657 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:12.553349    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:12.580686 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:12.580605    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph" podStartSLOduration=2.131836746 podStartE2EDuration="6.580589089s" totalImagesPullingTime="3.333582387s" totalInitContainerRuntime="1.115169956s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:12.579484016 +0000 UTC m=+303.904006114" watchObservedRunningTime="2026-07-26 18:17:12.580589089 +0000 UTC m=+303.905111186"
Jul 26 18:17:23.559853 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:23.559821    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-758747dbfc-mqvph"
Jul 26 18:17:26.063440 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.063401    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"]
Jul 26 18:17:26.072240 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.072215    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.077022 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.076990    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 26 18:17:26.077177 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.077060    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-rr6s2\""
Jul 26 18:17:26.077177 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.076997    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 26 18:17:26.077324 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.077305    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 26 18:17:26.079031 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.079009    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"]
Jul 26 18:17:26.143145 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.143108    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5twt\" (UniqueName: \"kubernetes.io/projected/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-kube-api-access-n5twt\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.143359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.143161    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-metrics-cert\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.143359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.143199    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-cert\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.143359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.143314    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-manager-config\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.246051 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.245092    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-metrics-cert\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.246051 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.245177    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-cert\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.246051 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.245259    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-manager-config\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.246051 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.245314    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5twt\" (UniqueName: \"kubernetes.io/projected/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-kube-api-access-n5twt\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.251053 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.247269    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-manager-config\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.251053 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.248940    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-cert\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.251866 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.251847    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-metrics-cert\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.255246 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.255218    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5twt\" (UniqueName: \"kubernetes.io/projected/0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5-kube-api-access-n5twt\") pod \"lws-controller-manager-5b76bfd8c9-nhhr6\" (UID: \"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5\") " pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.382591 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.382494    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:26.523684 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.523526    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"]
Jul 26 18:17:26.526471 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:17:26.526443    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e3b51c7_ff6b_47d5_8c03_158ed9c0bcd5.slice/crio-d4b1f66302977065b2e20ddce9521aadac7c7ca26b509d5a87dcb20138b4fa36 WatchSource:0}: Error finding container d4b1f66302977065b2e20ddce9521aadac7c7ca26b509d5a87dcb20138b4fa36: Status 404 returned error can't find the container with id d4b1f66302977065b2e20ddce9521aadac7c7ca26b509d5a87dcb20138b4fa36
Jul 26 18:17:26.616461 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:26.616428    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6" event={"ID":"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5","Type":"ContainerStarted","Data":"d4b1f66302977065b2e20ddce9521aadac7c7ca26b509d5a87dcb20138b4fa36"}
Jul 26 18:17:28.627804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:28.627772    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6" event={"ID":"0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5","Type":"ContainerStarted","Data":"5af9a96c11e2b2bff0c655cf6e02b5fd4ea552f6fe7dd775853bd2c2b811aa97"}
Jul 26 18:17:28.628199 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:28.627890    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:28.646063 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:28.646009    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6" podStartSLOduration=1.259939535 podStartE2EDuration="2.645991542s" totalImagesPullingTime="1.386052007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:28.644936572 +0000 UTC m=+319.969458691" watchObservedRunningTime="2026-07-26 18:17:28.645991542 +0000 UTC m=+319.970513638"
Jul 26 18:17:36.280647 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.280594    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"]
Jul 26 18:17:36.283198 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.283176    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.287923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.287901    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 26 18:17:36.289101 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.289079    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 26 18:17:36.289274 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.289217    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 26 18:17:36.289354 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.289091    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 26 18:17:36.289354 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.289141    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-659n7\""
Jul 26 18:17:36.307502 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.307471    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"]
Jul 26 18:17:36.337914 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.337885    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89307c3c-e972-476b-963c-4df2e54d2f92-tmp\") pod \"kube-auth-proxy-5c7c67d4f6-b8krv\" (UID: \"89307c3c-e972-476b-963c-4df2e54d2f92\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.338082 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.337924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89307c3c-e972-476b-963c-4df2e54d2f92-tls-certs\") pod \"kube-auth-proxy-5c7c67d4f6-b8krv\" (UID: \"89307c3c-e972-476b-963c-4df2e54d2f92\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.338082 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.337954    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppg6j\" (UniqueName: \"kubernetes.io/projected/89307c3c-e972-476b-963c-4df2e54d2f92-kube-api-access-ppg6j\") pod \"kube-auth-proxy-5c7c67d4f6-b8krv\" (UID: \"89307c3c-e972-476b-963c-4df2e54d2f92\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.439285 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.439256    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89307c3c-e972-476b-963c-4df2e54d2f92-tmp\") pod \"kube-auth-proxy-5c7c67d4f6-b8krv\" (UID: \"89307c3c-e972-476b-963c-4df2e54d2f92\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.439285 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.439293    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89307c3c-e972-476b-963c-4df2e54d2f92-tls-certs\") pod \"kube-auth-proxy-5c7c67d4f6-b8krv\" (UID: \"89307c3c-e972-476b-963c-4df2e54d2f92\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.439485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.439315    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppg6j\" (UniqueName: \"kubernetes.io/projected/89307c3c-e972-476b-963c-4df2e54d2f92-kube-api-access-ppg6j\") pod \"kube-auth-proxy-5c7c67d4f6-b8krv\" (UID: \"89307c3c-e972-476b-963c-4df2e54d2f92\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.441656 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.441609    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89307c3c-e972-476b-963c-4df2e54d2f92-tmp\") pod \"kube-auth-proxy-5c7c67d4f6-b8krv\" (UID: \"89307c3c-e972-476b-963c-4df2e54d2f92\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.441858 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.441836    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89307c3c-e972-476b-963c-4df2e54d2f92-tls-certs\") pod \"kube-auth-proxy-5c7c67d4f6-b8krv\" (UID: \"89307c3c-e972-476b-963c-4df2e54d2f92\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.448919 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.448890    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppg6j\" (UniqueName: \"kubernetes.io/projected/89307c3c-e972-476b-963c-4df2e54d2f92-kube-api-access-ppg6j\") pod \"kube-auth-proxy-5c7c67d4f6-b8krv\" (UID: \"89307c3c-e972-476b-963c-4df2e54d2f92\") " pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.593548 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.593459    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"
Jul 26 18:17:36.777536 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:36.777510    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv"]
Jul 26 18:17:36.780158 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:17:36.780134    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89307c3c_e972_476b_963c_4df2e54d2f92.slice/crio-0b10061844b6cc60395c09840f4c4e7cfddf64f9ae8d904cdaf6daefc06cd1ca WatchSource:0}: Error finding container 0b10061844b6cc60395c09840f4c4e7cfddf64f9ae8d904cdaf6daefc06cd1ca: Status 404 returned error can't find the container with id 0b10061844b6cc60395c09840f4c4e7cfddf64f9ae8d904cdaf6daefc06cd1ca
Jul 26 18:17:37.681334 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:37.681288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv" event={"ID":"89307c3c-e972-476b-963c-4df2e54d2f92","Type":"ContainerStarted","Data":"0b10061844b6cc60395c09840f4c4e7cfddf64f9ae8d904cdaf6daefc06cd1ca"}
Jul 26 18:17:39.634390 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:39.634306    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5b76bfd8c9-nhhr6"
Jul 26 18:17:39.692612 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:39.692577    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv" event={"ID":"89307c3c-e972-476b-963c-4df2e54d2f92","Type":"ContainerStarted","Data":"2560f8356e154dbfc278c66283783873f4d2d9b4891ea401ab53bb675ea75ccf"}
Jul 26 18:17:39.727831 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:17:39.727777    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c7c67d4f6-b8krv" podStartSLOduration=1.262530027 podStartE2EDuration="3.727758884s" totalImagesPullingTime="2.465228857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:17:39.725506464 +0000 UTC m=+331.050028583" watchObservedRunningTime="2026-07-26 18:17:39.727758884 +0000 UTC m=+331.052281002"
Jul 26 18:19:18.702198 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.702164    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"]
Jul 26 18:19:18.705649 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.705612    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:18.708470 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.708443    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 26 18:19:18.708586 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.708532    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 26 18:19:18.708586 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.708541    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-rvvv5\""
Jul 26 18:19:18.708586 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.708537    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 26 18:19:18.709792 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.709777    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 26 18:19:18.714980 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.714958    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"]
Jul 26 18:19:18.752998 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.752959    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtj2q\" (UniqueName: \"kubernetes.io/projected/504040e5-5cb4-4fc9-a037-ed70147096b7-kube-api-access-xtj2q\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:18.752998 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.753004    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/504040e5-5cb4-4fc9-a037-ed70147096b7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:18.753219 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.753032    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/504040e5-5cb4-4fc9-a037-ed70147096b7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:18.854165 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.854126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/504040e5-5cb4-4fc9-a037-ed70147096b7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:18.854165 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.854173    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/504040e5-5cb4-4fc9-a037-ed70147096b7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:18.854391 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.854237    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtj2q\" (UniqueName: \"kubernetes.io/projected/504040e5-5cb4-4fc9-a037-ed70147096b7-kube-api-access-xtj2q\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:18.854391 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:19:18.854285    2559 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 26 18:19:18.854391 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:19:18.854358    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/504040e5-5cb4-4fc9-a037-ed70147096b7-plugin-serving-cert podName:504040e5-5cb4-4fc9-a037-ed70147096b7 nodeName:}" failed. No retries permitted until 2026-07-26 18:19:19.354339447 +0000 UTC m=+430.678861525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/504040e5-5cb4-4fc9-a037-ed70147096b7-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-spb9v" (UID: "504040e5-5cb4-4fc9-a037-ed70147096b7") : secret "plugin-serving-cert" not found
Jul 26 18:19:18.854918 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.854900    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/504040e5-5cb4-4fc9-a037-ed70147096b7-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:18.863860 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:18.863827    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtj2q\" (UniqueName: \"kubernetes.io/projected/504040e5-5cb4-4fc9-a037-ed70147096b7-kube-api-access-xtj2q\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:19.357553 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:19.357513    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/504040e5-5cb4-4fc9-a037-ed70147096b7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:19.359987 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:19.359959    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/504040e5-5cb4-4fc9-a037-ed70147096b7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-spb9v\" (UID: \"504040e5-5cb4-4fc9-a037-ed70147096b7\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:19.615945 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:19.615857    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"
Jul 26 18:19:19.752646 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:19.752591    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v"]
Jul 26 18:19:19.755289 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:19:19.755261    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod504040e5_5cb4_4fc9_a037_ed70147096b7.slice/crio-71e60ae84b9b945a9af38b35b059d75d1e32a96341feffce65d9715f3c5465c7 WatchSource:0}: Error finding container 71e60ae84b9b945a9af38b35b059d75d1e32a96341feffce65d9715f3c5465c7: Status 404 returned error can't find the container with id 71e60ae84b9b945a9af38b35b059d75d1e32a96341feffce65d9715f3c5465c7
Jul 26 18:19:20.153758 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:20.153728    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v" event={"ID":"504040e5-5cb4-4fc9-a037-ed70147096b7","Type":"ContainerStarted","Data":"71e60ae84b9b945a9af38b35b059d75d1e32a96341feffce65d9715f3c5465c7"}
Jul 26 18:19:29.831854 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.831820    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"]
Jul 26 18:19:29.859107 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.859074    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:29.861807 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.861758    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"]
Jul 26 18:19:29.862802 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.862774    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8rshk\""
Jul 26 18:19:29.935153 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.934892    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"]
Jul 26 18:19:29.935331 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:19:29.935297    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-75j8v], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s" podUID="58312c94-0856-4a4d-b5e1-307cb8d4b5db"
Jul 26 18:19:29.946636 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.946582    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"]
Jul 26 18:19:29.958540 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.958491    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58312c94-0856-4a4d-b5e1-307cb8d4b5db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gkx6s\" (UID: \"58312c94-0856-4a4d-b5e1-307cb8d4b5db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:29.958540 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.958535    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75j8v\" (UniqueName: \"kubernetes.io/projected/58312c94-0856-4a4d-b5e1-307cb8d4b5db-kube-api-access-75j8v\") pod \"kuadrant-operator-controller-manager-86878c769f-gkx6s\" (UID: \"58312c94-0856-4a4d-b5e1-307cb8d4b5db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:29.982001 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.981952    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"]
Jul 26 18:19:29.987069 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:29.987036    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:19:30.011701 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.011665    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"]
Jul 26 18:19:30.060533 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.059784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twtdp\" (UniqueName: \"kubernetes.io/projected/40864ec8-6f22-446b-bf10-3ce81702368d-kube-api-access-twtdp\") pod \"kuadrant-operator-controller-manager-86878c769f-29ft9\" (UID: \"40864ec8-6f22-446b-bf10-3ce81702368d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:19:30.060533 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.059894    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58312c94-0856-4a4d-b5e1-307cb8d4b5db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gkx6s\" (UID: \"58312c94-0856-4a4d-b5e1-307cb8d4b5db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:30.060533 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.059927    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75j8v\" (UniqueName: \"kubernetes.io/projected/58312c94-0856-4a4d-b5e1-307cb8d4b5db-kube-api-access-75j8v\") pod \"kuadrant-operator-controller-manager-86878c769f-gkx6s\" (UID: \"58312c94-0856-4a4d-b5e1-307cb8d4b5db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:30.060533 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.060000    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40864ec8-6f22-446b-bf10-3ce81702368d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-29ft9\" (UID: \"40864ec8-6f22-446b-bf10-3ce81702368d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:19:30.060533 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.060465    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58312c94-0856-4a4d-b5e1-307cb8d4b5db-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gkx6s\" (UID: \"58312c94-0856-4a4d-b5e1-307cb8d4b5db\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:30.068378 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:19:30.068348    2559 projected.go:196] Error preparing data for projected volume kube-api-access-75j8v for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-gkx6s" not found
Jul 26 18:19:30.068556 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:19:30.068443    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58312c94-0856-4a4d-b5e1-307cb8d4b5db-kube-api-access-75j8v podName:58312c94-0856-4a4d-b5e1-307cb8d4b5db nodeName:}" failed. No retries permitted until 2026-07-26 18:19:30.568420963 +0000 UTC m=+441.892943042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-75j8v" (UniqueName: "kubernetes.io/projected/58312c94-0856-4a4d-b5e1-307cb8d4b5db-kube-api-access-75j8v") pod "kuadrant-operator-controller-manager-86878c769f-gkx6s" (UID: "58312c94-0856-4a4d-b5e1-307cb8d4b5db") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-gkx6s" not found
Jul 26 18:19:30.160704 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.160591    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twtdp\" (UniqueName: \"kubernetes.io/projected/40864ec8-6f22-446b-bf10-3ce81702368d-kube-api-access-twtdp\") pod \"kuadrant-operator-controller-manager-86878c769f-29ft9\" (UID: \"40864ec8-6f22-446b-bf10-3ce81702368d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:19:30.160879 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.160792    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40864ec8-6f22-446b-bf10-3ce81702368d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-29ft9\" (UID: \"40864ec8-6f22-446b-bf10-3ce81702368d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:19:30.161216 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.161195    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40864ec8-6f22-446b-bf10-3ce81702368d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-29ft9\" (UID: \"40864ec8-6f22-446b-bf10-3ce81702368d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:19:30.172903 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.172869    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twtdp\" (UniqueName: \"kubernetes.io/projected/40864ec8-6f22-446b-bf10-3ce81702368d-kube-api-access-twtdp\") pod \"kuadrant-operator-controller-manager-86878c769f-29ft9\" (UID: \"40864ec8-6f22-446b-bf10-3ce81702368d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:19:30.210936 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.210903    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:30.216278 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:19:30.216245    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-gkx6s\" is forbidden: User \"system:node:ip-10-0-138-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-70.ec2.internal' and this object" podUID="58312c94-0856-4a4d-b5e1-307cb8d4b5db" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:30.223565 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.223529    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:30.226284 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:19:30.226244    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-gkx6s\" is forbidden: User \"system:node:ip-10-0-138-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-70.ec2.internal' and this object" podUID="58312c94-0856-4a4d-b5e1-307cb8d4b5db" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:30.300351 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.300317    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:19:30.362450 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.362410    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58312c94-0856-4a4d-b5e1-307cb8d4b5db-extensions-socket-volume\") pod \"58312c94-0856-4a4d-b5e1-307cb8d4b5db\" (UID: \"58312c94-0856-4a4d-b5e1-307cb8d4b5db\") "
Jul 26 18:19:30.362673 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.362656    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75j8v\" (UniqueName: \"kubernetes.io/projected/58312c94-0856-4a4d-b5e1-307cb8d4b5db-kube-api-access-75j8v\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:19:30.362781 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.362757    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58312c94-0856-4a4d-b5e1-307cb8d4b5db-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "58312c94-0856-4a4d-b5e1-307cb8d4b5db" (UID: "58312c94-0856-4a4d-b5e1-307cb8d4b5db"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:19:30.463944 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:30.463910    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58312c94-0856-4a4d-b5e1-307cb8d4b5db-extensions-socket-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:19:31.181873 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:31.181838    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58312c94-0856-4a4d-b5e1-307cb8d4b5db" path="/var/lib/kubelet/pods/58312c94-0856-4a4d-b5e1-307cb8d4b5db/volumes"
Jul 26 18:19:31.216147 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:31.216117    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:31.221732 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:19:31.221696    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-gkx6s\" is forbidden: User \"system:node:ip-10-0-138-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-70.ec2.internal' and this object" podUID="58312c94-0856-4a4d-b5e1-307cb8d4b5db" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gkx6s"
Jul 26 18:19:42.364894 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:42.364864    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"]
Jul 26 18:19:42.366789 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:19:42.366761    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40864ec8_6f22_446b_bf10_3ce81702368d.slice/crio-66c0c5559b4d9a7aafbaf3eea683e9648ddd15824b6808bc68c116f58903a96a WatchSource:0}: Error finding container 66c0c5559b4d9a7aafbaf3eea683e9648ddd15824b6808bc68c116f58903a96a: Status 404 returned error can't find the container with id 66c0c5559b4d9a7aafbaf3eea683e9648ddd15824b6808bc68c116f58903a96a
Jul 26 18:19:43.282961 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:43.282922    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v" event={"ID":"504040e5-5cb4-4fc9-a037-ed70147096b7","Type":"ContainerStarted","Data":"ff71c07d7aa87715656e9dc371b72c2b0c6b2d6214f384aecc679314650960d7"}
Jul 26 18:19:43.284353 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:43.284322    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9" event={"ID":"40864ec8-6f22-446b-bf10-3ce81702368d","Type":"ContainerStarted","Data":"66c0c5559b4d9a7aafbaf3eea683e9648ddd15824b6808bc68c116f58903a96a"}
Jul 26 18:19:43.305963 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:43.305890    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-spb9v" podStartSLOduration=2.4458802410000002 podStartE2EDuration="25.305873399s" totalImagesPullingTime="22.859993158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:43.304656769 +0000 UTC m=+454.629178864" watchObservedRunningTime="2026-07-26 18:19:43.305873399 +0000 UTC m=+454.630395496"
Jul 26 18:19:48.313902 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:48.313820    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9" event={"ID":"40864ec8-6f22-446b-bf10-3ce81702368d","Type":"ContainerStarted","Data":"380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60"}
Jul 26 18:19:48.314364 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:48.314060    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:19:48.339829 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:48.339773    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9" podStartSLOduration=13.703385325 podStartE2EDuration="19.339756637s" totalImagesPullingTime="5.636371312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:19:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:19:48.338270969 +0000 UTC m=+459.662793070" watchObservedRunningTime="2026-07-26 18:19:48.339756637 +0000 UTC m=+459.664278734"
Jul 26 18:19:59.321273 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:19:59.321240    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:20:21.017127 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:21.017089    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-569mm"]
Jul 26 18:20:21.076873 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:21.076830    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-569mm"]
Jul 26 18:20:21.077044 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:21.076932    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-569mm"
Jul 26 18:20:21.079977 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:21.079959    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4fj65\""
Jul 26 18:20:21.253868 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:21.253827    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbhfb\" (UniqueName: \"kubernetes.io/projected/bddbfaad-967f-46c8-a5f7-615df9cb9f50-kube-api-access-wbhfb\") pod \"authorino-75d556bd77-569mm\" (UID: \"bddbfaad-967f-46c8-a5f7-615df9cb9f50\") " pod="kuadrant-system/authorino-75d556bd77-569mm"
Jul 26 18:20:21.355238 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:21.355139    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbhfb\" (UniqueName: \"kubernetes.io/projected/bddbfaad-967f-46c8-a5f7-615df9cb9f50-kube-api-access-wbhfb\") pod \"authorino-75d556bd77-569mm\" (UID: \"bddbfaad-967f-46c8-a5f7-615df9cb9f50\") " pod="kuadrant-system/authorino-75d556bd77-569mm"
Jul 26 18:20:21.366899 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:21.366867    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbhfb\" (UniqueName: \"kubernetes.io/projected/bddbfaad-967f-46c8-a5f7-615df9cb9f50-kube-api-access-wbhfb\") pod \"authorino-75d556bd77-569mm\" (UID: \"bddbfaad-967f-46c8-a5f7-615df9cb9f50\") " pod="kuadrant-system/authorino-75d556bd77-569mm"
Jul 26 18:20:21.386651 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:21.386595    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-569mm"
Jul 26 18:20:21.523047 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:21.523019    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-569mm"]
Jul 26 18:20:21.525656 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:20:21.525628    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbddbfaad_967f_46c8_a5f7_615df9cb9f50.slice/crio-ac61ce3430434d446d5998dc4301d89f91753d72e220f5512d0bda683f919c6f WatchSource:0}: Error finding container ac61ce3430434d446d5998dc4301d89f91753d72e220f5512d0bda683f919c6f: Status 404 returned error can't find the container with id ac61ce3430434d446d5998dc4301d89f91753d72e220f5512d0bda683f919c6f
Jul 26 18:20:22.484368 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:22.484328    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-569mm" event={"ID":"bddbfaad-967f-46c8-a5f7-615df9cb9f50","Type":"ContainerStarted","Data":"ac61ce3430434d446d5998dc4301d89f91753d72e220f5512d0bda683f919c6f"}
Jul 26 18:20:25.500818 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:25.500784    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-569mm" event={"ID":"bddbfaad-967f-46c8-a5f7-615df9cb9f50","Type":"ContainerStarted","Data":"ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df"}
Jul 26 18:20:25.519836 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:25.519781    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-569mm" podStartSLOduration=2.496480124 podStartE2EDuration="5.519763527s" totalImagesPullingTime="3.023283403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:20:25.518685212 +0000 UTC m=+496.843207311" watchObservedRunningTime="2026-07-26 18:20:25.519763527 +0000 UTC m=+496.844285624"
Jul 26 18:20:55.624979 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:55.624946    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-569mm"]
Jul 26 18:20:55.625469 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:55.625129    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-569mm" podUID="bddbfaad-967f-46c8-a5f7-615df9cb9f50" containerName="authorino" containerID="cri-o://ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df" gracePeriod=30
Jul 26 18:20:55.877126 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:55.877062    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-569mm"
Jul 26 18:20:55.982174 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:55.982141    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbhfb\" (UniqueName: \"kubernetes.io/projected/bddbfaad-967f-46c8-a5f7-615df9cb9f50-kube-api-access-wbhfb\") pod \"bddbfaad-967f-46c8-a5f7-615df9cb9f50\" (UID: \"bddbfaad-967f-46c8-a5f7-615df9cb9f50\") "
Jul 26 18:20:55.984343 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:55.984310    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bddbfaad-967f-46c8-a5f7-615df9cb9f50-kube-api-access-wbhfb" (OuterVolumeSpecName: "kube-api-access-wbhfb") pod "bddbfaad-967f-46c8-a5f7-615df9cb9f50" (UID: "bddbfaad-967f-46c8-a5f7-615df9cb9f50"). InnerVolumeSpecName "kube-api-access-wbhfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:20:56.083719 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.083680    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbhfb\" (UniqueName: \"kubernetes.io/projected/bddbfaad-967f-46c8-a5f7-615df9cb9f50-kube-api-access-wbhfb\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:20:56.641437 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.641400    2559 generic.go:356] "Generic (PLEG): container finished" podID="bddbfaad-967f-46c8-a5f7-615df9cb9f50" containerID="ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df" exitCode=0
Jul 26 18:20:56.641906 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.641460    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-569mm"
Jul 26 18:20:56.641906 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.641487    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-569mm" event={"ID":"bddbfaad-967f-46c8-a5f7-615df9cb9f50","Type":"ContainerDied","Data":"ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df"}
Jul 26 18:20:56.641906 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.641529    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-569mm" event={"ID":"bddbfaad-967f-46c8-a5f7-615df9cb9f50","Type":"ContainerDied","Data":"ac61ce3430434d446d5998dc4301d89f91753d72e220f5512d0bda683f919c6f"}
Jul 26 18:20:56.641906 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.641552    2559 scope.go:117] "RemoveContainer" containerID="ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df"
Jul 26 18:20:56.650185 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.650168    2559 scope.go:117] "RemoveContainer" containerID="ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df"
Jul 26 18:20:56.650487 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:20:56.650466    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df\": container with ID starting with ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df not found: ID does not exist" containerID="ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df"
Jul 26 18:20:56.650529 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.650496    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df"} err="failed to get container status \"ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df\": rpc error: code = NotFound desc = could not find container \"ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df\": container with ID starting with ec99973e31ef4ef096fd28c0074fb90f51fa09d66402178f81fb4b36a98a23df not found: ID does not exist"
Jul 26 18:20:56.663136 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.663101    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-569mm"]
Jul 26 18:20:56.666333 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:56.666305    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-569mm"]
Jul 26 18:20:57.176841 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:20:57.176808    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bddbfaad-967f-46c8-a5f7-615df9cb9f50" path="/var/lib/kubelet/pods/bddbfaad-967f-46c8-a5f7-615df9cb9f50/volumes"
Jul 26 18:21:00.282759 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.282725    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-699866b499-gjkbc"]
Jul 26 18:21:00.283154 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.283082    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bddbfaad-967f-46c8-a5f7-615df9cb9f50" containerName="authorino"
Jul 26 18:21:00.283154 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.283093    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bddbfaad-967f-46c8-a5f7-615df9cb9f50" containerName="authorino"
Jul 26 18:21:00.283154 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.283152    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bddbfaad-967f-46c8-a5f7-615df9cb9f50" containerName="authorino"
Jul 26 18:21:00.288122 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.288098    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:00.293061 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:21:00.293007    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-controller-699866b499-gjkbc\" is forbidden: User \"system:node:ip-10-0-138-70.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-70.ec2.internal' and this object" podUID="3cadf480-6bdc-4b0e-ba3c-a9517755e46c" pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:00.293061 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:21:00.293031    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-controller-dockercfg-8srmb\" is forbidden: User \"system:node:ip-10-0-138-70.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-70.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8srmb\"" type="*v1.Secret"
Jul 26 18:21:00.293271 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:21:00.293154    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"maas-parameters\" is forbidden: User \"system:node:ip-10-0-138-70.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-70.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"maas-parameters\"" type="*v1.ConfigMap"
Jul 26 18:21:00.293271 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:21:00.293177    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-138-70.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-138-70.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" type="*v1.Secret"
Jul 26 18:21:00.309920 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.309885    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-699866b499-gjkbc"]
Jul 26 18:21:00.424631 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.424575    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-webhook-cert\") pod \"maas-controller-699866b499-gjkbc\" (UID: \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\") " pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:00.424817 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.424652    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z4ng\" (UniqueName: \"kubernetes.io/projected/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-kube-api-access-5z4ng\") pod \"maas-controller-699866b499-gjkbc\" (UID: \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\") " pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:00.488160 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.488124    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8494c84d99-7qcl6"]
Jul 26 18:21:00.491111 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.491094    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:00.523201 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.523160    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8494c84d99-7qcl6"]
Jul 26 18:21:00.525520 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.525496    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-webhook-cert\") pod \"maas-controller-699866b499-gjkbc\" (UID: \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\") " pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:00.525606 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.525535    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z4ng\" (UniqueName: \"kubernetes.io/projected/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-kube-api-access-5z4ng\") pod \"maas-controller-699866b499-gjkbc\" (UID: \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\") " pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:00.543095 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.543021    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z4ng\" (UniqueName: \"kubernetes.io/projected/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-kube-api-access-5z4ng\") pod \"maas-controller-699866b499-gjkbc\" (UID: \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\") " pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:00.627102 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.627007    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc4fv\" (UniqueName: \"kubernetes.io/projected/6b507195-25ae-436f-a2a5-7815573d9685-kube-api-access-gc4fv\") pod \"maas-controller-8494c84d99-7qcl6\" (UID: \"6b507195-25ae-436f-a2a5-7815573d9685\") " pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:00.627300 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.627166    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b507195-25ae-436f-a2a5-7815573d9685-webhook-cert\") pod \"maas-controller-8494c84d99-7qcl6\" (UID: \"6b507195-25ae-436f-a2a5-7815573d9685\") " pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:00.728690 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.728645    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b507195-25ae-436f-a2a5-7815573d9685-webhook-cert\") pod \"maas-controller-8494c84d99-7qcl6\" (UID: \"6b507195-25ae-436f-a2a5-7815573d9685\") " pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:00.728859 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.728716    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc4fv\" (UniqueName: \"kubernetes.io/projected/6b507195-25ae-436f-a2a5-7815573d9685-kube-api-access-gc4fv\") pod \"maas-controller-8494c84d99-7qcl6\" (UID: \"6b507195-25ae-436f-a2a5-7815573d9685\") " pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:00.751538 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:00.751505    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc4fv\" (UniqueName: \"kubernetes.io/projected/6b507195-25ae-436f-a2a5-7815573d9685-kube-api-access-gc4fv\") pod \"maas-controller-8494c84d99-7qcl6\" (UID: \"6b507195-25ae-436f-a2a5-7815573d9685\") " pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:01.246359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:01.246323    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 18:21:01.251411 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:01.251381    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b507195-25ae-436f-a2a5-7815573d9685-webhook-cert\") pod \"maas-controller-8494c84d99-7qcl6\" (UID: \"6b507195-25ae-436f-a2a5-7815573d9685\") " pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:01.258524 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:01.258493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-webhook-cert\") pod \"maas-controller-699866b499-gjkbc\" (UID: \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\") " pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:01.784158 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:01.784131    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8srmb\""
Jul 26 18:21:01.789973 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:01.789952    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:01.791160 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:01.791140    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:01.885983 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:01.885884    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 18:21:01.940680 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:01.940655    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-699866b499-gjkbc"]
Jul 26 18:21:01.942060 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:21:01.942024    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cadf480_6bdc_4b0e_ba3c_a9517755e46c.slice/crio-efec3ae90211ccdd043588837852a1b420df64081068b07dcf9c961e1438298d WatchSource:0}: Error finding container efec3ae90211ccdd043588837852a1b420df64081068b07dcf9c961e1438298d: Status 404 returned error can't find the container with id efec3ae90211ccdd043588837852a1b420df64081068b07dcf9c961e1438298d
Jul 26 18:21:01.966875 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:01.966844    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8494c84d99-7qcl6"]
Jul 26 18:21:01.969825 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:21:01.969795    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b507195_25ae_436f_a2a5_7815573d9685.slice/crio-de7a88942cd384f766d304155f7dd5fbf11cc3cc4f4ac9e6af499a468099f459 WatchSource:0}: Error finding container de7a88942cd384f766d304155f7dd5fbf11cc3cc4f4ac9e6af499a468099f459: Status 404 returned error can't find the container with id de7a88942cd384f766d304155f7dd5fbf11cc3cc4f4ac9e6af499a468099f459
Jul 26 18:21:02.675723 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:02.675685    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-7qcl6" event={"ID":"6b507195-25ae-436f-a2a5-7815573d9685","Type":"ContainerStarted","Data":"de7a88942cd384f766d304155f7dd5fbf11cc3cc4f4ac9e6af499a468099f459"}
Jul 26 18:21:02.677347 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:02.677257    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699866b499-gjkbc" event={"ID":"3cadf480-6bdc-4b0e-ba3c-a9517755e46c","Type":"ContainerStarted","Data":"efec3ae90211ccdd043588837852a1b420df64081068b07dcf9c961e1438298d"}
Jul 26 18:21:04.689943 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:04.689831    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699866b499-gjkbc" event={"ID":"3cadf480-6bdc-4b0e-ba3c-a9517755e46c","Type":"ContainerStarted","Data":"57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510"}
Jul 26 18:21:04.690386 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:04.690028    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:04.691453 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:04.691430    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-7qcl6" event={"ID":"6b507195-25ae-436f-a2a5-7815573d9685","Type":"ContainerStarted","Data":"9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d"}
Jul 26 18:21:04.691559 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:04.691482    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:04.712028 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:04.711978    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-699866b499-gjkbc" podStartSLOduration=2.23305043 podStartE2EDuration="4.711960961s" totalImagesPullingTime="2.478910531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:21:04.711344141 +0000 UTC m=+536.035866238" watchObservedRunningTime="2026-07-26 18:21:04.711960961 +0000 UTC m=+536.036483057"
Jul 26 18:21:04.731945 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:04.731890    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8494c84d99-7qcl6" podStartSLOduration=2.274764761 podStartE2EDuration="4.731866115s" totalImagesPullingTime="2.457101354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:21:04.72987985 +0000 UTC m=+536.054401962" watchObservedRunningTime="2026-07-26 18:21:04.731866115 +0000 UTC m=+536.056388212"
Jul 26 18:21:15.699011 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:15.698975    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:21:15.699434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:15.699045    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:15.769017 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:15.768981    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-699866b499-gjkbc"]
Jul 26 18:21:15.769245 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:15.769195    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-699866b499-gjkbc" podUID="3cadf480-6bdc-4b0e-ba3c-a9517755e46c" containerName="manager" containerID="cri-o://57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510" gracePeriod=30
Jul 26 18:21:16.021526 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.021502    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:16.065182 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.065144    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-webhook-cert\") pod \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\" (UID: \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\") "
Jul 26 18:21:16.065383 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.065188    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5z4ng\" (UniqueName: \"kubernetes.io/projected/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-kube-api-access-5z4ng\") pod \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\" (UID: \"3cadf480-6bdc-4b0e-ba3c-a9517755e46c\") "
Jul 26 18:21:16.067460 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.067432    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "3cadf480-6bdc-4b0e-ba3c-a9517755e46c" (UID: "3cadf480-6bdc-4b0e-ba3c-a9517755e46c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:21:16.067557 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.067496    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-kube-api-access-5z4ng" (OuterVolumeSpecName: "kube-api-access-5z4ng") pod "3cadf480-6bdc-4b0e-ba3c-a9517755e46c" (UID: "3cadf480-6bdc-4b0e-ba3c-a9517755e46c"). InnerVolumeSpecName "kube-api-access-5z4ng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:21:16.165952 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.165917    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-webhook-cert\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:21:16.165952 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.165950    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5z4ng\" (UniqueName: \"kubernetes.io/projected/3cadf480-6bdc-4b0e-ba3c-a9517755e46c-kube-api-access-5z4ng\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:21:16.752092 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.752055    2559 generic.go:356] "Generic (PLEG): container finished" podID="3cadf480-6bdc-4b0e-ba3c-a9517755e46c" containerID="57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510" exitCode=0
Jul 26 18:21:16.752494 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.752110    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699866b499-gjkbc"
Jul 26 18:21:16.752494 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.752146    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699866b499-gjkbc" event={"ID":"3cadf480-6bdc-4b0e-ba3c-a9517755e46c","Type":"ContainerDied","Data":"57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510"}
Jul 26 18:21:16.752494 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.752207    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699866b499-gjkbc" event={"ID":"3cadf480-6bdc-4b0e-ba3c-a9517755e46c","Type":"ContainerDied","Data":"efec3ae90211ccdd043588837852a1b420df64081068b07dcf9c961e1438298d"}
Jul 26 18:21:16.752494 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.752238    2559 scope.go:117] "RemoveContainer" containerID="57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510"
Jul 26 18:21:16.761389 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.761370    2559 scope.go:117] "RemoveContainer" containerID="57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510"
Jul 26 18:21:16.761711 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:21:16.761685    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510\": container with ID starting with 57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510 not found: ID does not exist" containerID="57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510"
Jul 26 18:21:16.761803 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.761719    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510"} err="failed to get container status \"57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510\": rpc error: code = NotFound desc = could not find container \"57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510\": container with ID starting with 57acc8be93c16af632d4e181cc68c7e66bb33694b8d28d589487f9b106781510 not found: ID does not exist"
Jul 26 18:21:16.774787 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.774758    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-699866b499-gjkbc"]
Jul 26 18:21:16.778592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:16.778566    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-699866b499-gjkbc"]
Jul 26 18:21:17.176094 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:17.176064    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cadf480-6bdc-4b0e-ba3c-a9517755e46c" path="/var/lib/kubelet/pods/3cadf480-6bdc-4b0e-ba3c-a9517755e46c/volumes"
Jul 26 18:21:59.904891 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.904854    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"]
Jul 26 18:21:59.905466 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.905407    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cadf480-6bdc-4b0e-ba3c-a9517755e46c" containerName="manager"
Jul 26 18:21:59.905466 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.905427    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cadf480-6bdc-4b0e-ba3c-a9517755e46c" containerName="manager"
Jul 26 18:21:59.905579 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.905552    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cadf480-6bdc-4b0e-ba3c-a9517755e46c" containerName="manager"
Jul 26 18:21:59.912822 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.912797    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:21:59.915967 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.915943    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 26 18:21:59.916422 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.916375    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 26 18:21:59.916551 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.916230    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 26 18:21:59.916809 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.916775    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pxsw8\""
Jul 26 18:21:59.920331 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.920308    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"]
Jul 26 18:21:59.965084 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.965047    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80f6993d-e8bf-424d-a64e-72b33c31c5e1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:21:59.965258 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.965117    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:21:59.965258 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.965146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:21:59.965258 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.965211    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2h9h\" (UniqueName: \"kubernetes.io/projected/80f6993d-e8bf-424d-a64e-72b33c31c5e1-kube-api-access-v2h9h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:21:59.965258 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.965233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:21:59.965392 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:21:59.965284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.066286 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.066246    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.066458 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.066320    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.066458 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.066381    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80f6993d-e8bf-424d-a64e-72b33c31c5e1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.066458 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.066447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.066611 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.066471    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.066611 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.066532    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2h9h\" (UniqueName: \"kubernetes.io/projected/80f6993d-e8bf-424d-a64e-72b33c31c5e1-kube-api-access-v2h9h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.066748 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.066685    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.066891 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.066868    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.066953 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.066898    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.068747 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.068720    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80f6993d-e8bf-424d-a64e-72b33c31c5e1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.068898 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.068876    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80f6993d-e8bf-424d-a64e-72b33c31c5e1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.075021 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.074996    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2h9h\" (UniqueName: \"kubernetes.io/projected/80f6993d-e8bf-424d-a64e-72b33c31c5e1-kube-api-access-v2h9h\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9\" (UID: \"80f6993d-e8bf-424d-a64e-72b33c31c5e1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.227076 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.227026    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:00.372649 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.372600    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"]
Jul 26 18:22:00.375153 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:22:00.375123    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f6993d_e8bf_424d_a64e_72b33c31c5e1.slice/crio-c272ad70bd5753cf23a7c740d8dc854c55c1ac3f68d402feaaf3787bd7b29e5a WatchSource:0}: Error finding container c272ad70bd5753cf23a7c740d8dc854c55c1ac3f68d402feaaf3787bd7b29e5a: Status 404 returned error can't find the container with id c272ad70bd5753cf23a7c740d8dc854c55c1ac3f68d402feaaf3787bd7b29e5a
Jul 26 18:22:00.970415 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:00.970316    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9" event={"ID":"80f6993d-e8bf-424d-a64e-72b33c31c5e1","Type":"ContainerStarted","Data":"c272ad70bd5753cf23a7c740d8dc854c55c1ac3f68d402feaaf3787bd7b29e5a"}
Jul 26 18:22:01.976093 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:01.976062    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9" event={"ID":"80f6993d-e8bf-424d-a64e-72b33c31c5e1","Type":"ContainerStarted","Data":"989531a326ff7e1e55e8ddb5fd269164aaf8b42a028673503e396a52f6be3834"}
Jul 26 18:22:01.997155 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:01.997099    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9" podStartSLOduration=1.941069749 podStartE2EDuration="2.997079648s" totalImagesPullingTime="1.056009899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:01.995189774 +0000 UTC m=+593.319711871" watchObservedRunningTime="2026-07-26 18:22:01.997079648 +0000 UTC m=+593.321601744"
Jul 26 18:22:09.408931 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.408839    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"]
Jul 26 18:22:09.413659 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.413637    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.416536 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.416512    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 26 18:22:09.423126 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.423100    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"]
Jul 26 18:22:09.555449 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.555413    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.555674 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.555474    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.555674 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.555554    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.555674 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.555634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50c5641b-ad19-4b2f-811a-15235db96773-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.555849 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.555707    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.555849 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.555745    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdgzw\" (UniqueName: \"kubernetes.io/projected/50c5641b-ad19-4b2f-811a-15235db96773-kube-api-access-mdgzw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.656790 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.656753    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50c5641b-ad19-4b2f-811a-15235db96773-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.656983 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.656812    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.656983 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.656836    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdgzw\" (UniqueName: \"kubernetes.io/projected/50c5641b-ad19-4b2f-811a-15235db96773-kube-api-access-mdgzw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.656983 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.656873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.656983 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.656908    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.656983 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.656928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.657296 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.657261    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.657296 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.657286    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.657387 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.657314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.659300 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.659242    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/50c5641b-ad19-4b2f-811a-15235db96773-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.659397 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.659349    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/50c5641b-ad19-4b2f-811a-15235db96773-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.672272 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.672228    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdgzw\" (UniqueName: \"kubernetes.io/projected/50c5641b-ad19-4b2f-811a-15235db96773-kube-api-access-mdgzw\") pod \"e2e-distinct-simulated-kserve-59975d69f9-f5xrl\" (UID: \"50c5641b-ad19-4b2f-811a-15235db96773\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.724595 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.724463    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:09.753175 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.753142    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:22:09.754511 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.754481    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:22:09.879785 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:09.879747    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"]
Jul 26 18:22:09.882387 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:22:09.882358    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50c5641b_ad19_4b2f_811a_15235db96773.slice/crio-38c5dc631d360cdbe3a2e6a33040494aa1552dd51a573dcc43bc3bb0117e7b43 WatchSource:0}: Error finding container 38c5dc631d360cdbe3a2e6a33040494aa1552dd51a573dcc43bc3bb0117e7b43: Status 404 returned error can't find the container with id 38c5dc631d360cdbe3a2e6a33040494aa1552dd51a573dcc43bc3bb0117e7b43
Jul 26 18:22:10.014112 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:10.014080    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl" event={"ID":"50c5641b-ad19-4b2f-811a-15235db96773","Type":"ContainerStarted","Data":"38c5dc631d360cdbe3a2e6a33040494aa1552dd51a573dcc43bc3bb0117e7b43"}
Jul 26 18:22:10.227205 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:10.227155    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:10.227389 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:10.227263    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:10.240396 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:10.240369    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:11.020533 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:11.020495    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl" event={"ID":"50c5641b-ad19-4b2f-811a-15235db96773","Type":"ContainerStarted","Data":"03d24b4fc814823bfe2e248287b659fa15e4a00f0284801d47c19bd8b5d323fd"}
Jul 26 18:22:11.032024 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:11.031996    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9"
Jul 26 18:22:11.040165 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:11.040117    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl" podStartSLOduration=1.873247047 podStartE2EDuration="2.040101234s" totalImagesPullingTime="166.854187ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:22:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:11.039000884 +0000 UTC m=+602.363522993" watchObservedRunningTime="2026-07-26 18:22:11.040101234 +0000 UTC m=+602.364623382"
Jul 26 18:22:19.724996 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:19.724947    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:19.725419 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:19.725028    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:19.737717 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:19.737689    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:20.074417 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:20.074331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-f5xrl"
Jul 26 18:22:46.902999 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:46.902915    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"]
Jul 26 18:22:46.906664 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:46.906644    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:46.909605 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:46.909586    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 26 18:22:46.915982 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:46.915959    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"]
Jul 26 18:22:47.001008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.000970    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.001008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.001024    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.001219 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.001152    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93d6aaab-abbf-41af-bce4-91869932c551-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.001219 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.001183    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4d9g\" (UniqueName: \"kubernetes.io/projected/93d6aaab-abbf-41af-bce4-91869932c551-kube-api-access-q4d9g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.001289 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.001235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.001289 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.001266    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.102227 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.102185    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.102413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.102235    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.102413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.102296    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93d6aaab-abbf-41af-bce4-91869932c551-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.102413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.102316    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4d9g\" (UniqueName: \"kubernetes.io/projected/93d6aaab-abbf-41af-bce4-91869932c551-kube-api-access-q4d9g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.102413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.102361    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.102413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.102399    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.102780 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.102751    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.102942 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.102812    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.102942 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.102832    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.104715 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.104692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93d6aaab-abbf-41af-bce4-91869932c551-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.105070 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.105047    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93d6aaab-abbf-41af-bce4-91869932c551-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.110819 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.110782    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4d9g\" (UniqueName: \"kubernetes.io/projected/93d6aaab-abbf-41af-bce4-91869932c551-kube-api-access-q4d9g\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ch4nr\" (UID: \"93d6aaab-abbf-41af-bce4-91869932c551\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.218222 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.218181    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:47.362982 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.362945    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"]
Jul 26 18:22:47.366362 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:22:47.366328    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d6aaab_abbf_41af_bce4_91869932c551.slice/crio-a47e0e6d029817f827b43ad295f463f7c0de7edb24c0e9cbae508ab416fa5993 WatchSource:0}: Error finding container a47e0e6d029817f827b43ad295f463f7c0de7edb24c0e9cbae508ab416fa5993: Status 404 returned error can't find the container with id a47e0e6d029817f827b43ad295f463f7c0de7edb24c0e9cbae508ab416fa5993
Jul 26 18:22:47.367860 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:47.367836    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:22:48.200220 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:48.200183    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr" event={"ID":"93d6aaab-abbf-41af-bce4-91869932c551","Type":"ContainerStarted","Data":"56e8bc2cff0f8f112d10a11a1a558bed7845e8e08aed08de8612ebe8a5b10a77"}
Jul 26 18:22:48.200662 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:48.200238    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr" event={"ID":"93d6aaab-abbf-41af-bce4-91869932c551","Type":"ContainerStarted","Data":"a47e0e6d029817f827b43ad295f463f7c0de7edb24c0e9cbae508ab416fa5993"}
Jul 26 18:22:48.220603 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:48.220544    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr" podStartSLOduration=2.053707074 podStartE2EDuration="2.220525935s" totalImagesPullingTime="166.818861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:22:48.219391601 +0000 UTC m=+639.543913699" watchObservedRunningTime="2026-07-26 18:22:48.220525935 +0000 UTC m=+639.545048032"
Jul 26 18:22:57.218735 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:57.218688    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:57.219231 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:57.218772    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:57.231362 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:57.231326    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:22:57.254365 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:22:57.254331    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ch4nr"
Jul 26 18:24:49.409025 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.408991    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8494c84d99-7qcl6"]
Jul 26 18:24:49.409707 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.409181    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8494c84d99-7qcl6" podUID="6b507195-25ae-436f-a2a5-7815573d9685" containerName="manager" containerID="cri-o://9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d" gracePeriod=30
Jul 26 18:24:49.662065 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.661998    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:24:49.765218 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.765182    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b507195-25ae-436f-a2a5-7815573d9685-webhook-cert\") pod \"6b507195-25ae-436f-a2a5-7815573d9685\" (UID: \"6b507195-25ae-436f-a2a5-7815573d9685\") "
Jul 26 18:24:49.765390 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.765327    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gc4fv\" (UniqueName: \"kubernetes.io/projected/6b507195-25ae-436f-a2a5-7815573d9685-kube-api-access-gc4fv\") pod \"6b507195-25ae-436f-a2a5-7815573d9685\" (UID: \"6b507195-25ae-436f-a2a5-7815573d9685\") "
Jul 26 18:24:49.767415 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.767382    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b507195-25ae-436f-a2a5-7815573d9685-kube-api-access-gc4fv" (OuterVolumeSpecName: "kube-api-access-gc4fv") pod "6b507195-25ae-436f-a2a5-7815573d9685" (UID: "6b507195-25ae-436f-a2a5-7815573d9685"). InnerVolumeSpecName "kube-api-access-gc4fv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:24:49.767545 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.767414    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b507195-25ae-436f-a2a5-7815573d9685-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6b507195-25ae-436f-a2a5-7815573d9685" (UID: "6b507195-25ae-436f-a2a5-7815573d9685"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:24:49.787691 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.787658    2559 generic.go:356] "Generic (PLEG): container finished" podID="6b507195-25ae-436f-a2a5-7815573d9685" containerID="9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d" exitCode=0
Jul 26 18:24:49.787871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.787703    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-7qcl6"
Jul 26 18:24:49.787871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.787716    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-7qcl6" event={"ID":"6b507195-25ae-436f-a2a5-7815573d9685","Type":"ContainerDied","Data":"9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d"}
Jul 26 18:24:49.787871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.787757    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-7qcl6" event={"ID":"6b507195-25ae-436f-a2a5-7815573d9685","Type":"ContainerDied","Data":"de7a88942cd384f766d304155f7dd5fbf11cc3cc4f4ac9e6af499a468099f459"}
Jul 26 18:24:49.787871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.787781    2559 scope.go:117] "RemoveContainer" containerID="9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d"
Jul 26 18:24:49.797274 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.797257    2559 scope.go:117] "RemoveContainer" containerID="9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d"
Jul 26 18:24:49.797594 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:24:49.797573    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d\": container with ID starting with 9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d not found: ID does not exist" containerID="9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d"
Jul 26 18:24:49.797682 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.797609    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d"} err="failed to get container status \"9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d\": rpc error: code = NotFound desc = could not find container \"9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d\": container with ID starting with 9d6bcf91e0a73f9d3ead42fa1ff80ecf0caf837d485545b88b1a8a29e282a12d not found: ID does not exist"
Jul 26 18:24:49.810609 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.810572    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8494c84d99-7qcl6"]
Jul 26 18:24:49.814748 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.814718    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8494c84d99-7qcl6"]
Jul 26 18:24:49.866711 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.866671    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gc4fv\" (UniqueName: \"kubernetes.io/projected/6b507195-25ae-436f-a2a5-7815573d9685-kube-api-access-gc4fv\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:24:49.866711 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:49.866704    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6b507195-25ae-436f-a2a5-7815573d9685-webhook-cert\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:24:51.176702 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:24:51.176667    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b507195-25ae-436f-a2a5-7815573d9685" path="/var/lib/kubelet/pods/6b507195-25ae-436f-a2a5-7815573d9685/volumes"
Jul 26 18:25:50.571770 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.571689    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8494c84d99-nxrx7"]
Jul 26 18:25:50.572234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.572072    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b507195-25ae-436f-a2a5-7815573d9685" containerName="manager"
Jul 26 18:25:50.572234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.572083    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b507195-25ae-436f-a2a5-7815573d9685" containerName="manager"
Jul 26 18:25:50.572234 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.572175    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b507195-25ae-436f-a2a5-7815573d9685" containerName="manager"
Jul 26 18:25:50.574879 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.574860    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:25:50.577872 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.577848    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 18:25:50.578003 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.577853    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 18:25:50.579424 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.579405    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8srmb\""
Jul 26 18:25:50.584133 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.584108    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8494c84d99-nxrx7"]
Jul 26 18:25:50.653241 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.653200    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7a12aa8-7724-43c9-b8c8-5256d8c28778-webhook-cert\") pod \"maas-controller-8494c84d99-nxrx7\" (UID: \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\") " pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:25:50.653428 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.653260    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th8ch\" (UniqueName: \"kubernetes.io/projected/f7a12aa8-7724-43c9-b8c8-5256d8c28778-kube-api-access-th8ch\") pod \"maas-controller-8494c84d99-nxrx7\" (UID: \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\") " pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:25:50.754689 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.754654    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th8ch\" (UniqueName: \"kubernetes.io/projected/f7a12aa8-7724-43c9-b8c8-5256d8c28778-kube-api-access-th8ch\") pod \"maas-controller-8494c84d99-nxrx7\" (UID: \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\") " pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:25:50.754897 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.754825    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7a12aa8-7724-43c9-b8c8-5256d8c28778-webhook-cert\") pod \"maas-controller-8494c84d99-nxrx7\" (UID: \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\") " pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:25:50.757327 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.757303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7a12aa8-7724-43c9-b8c8-5256d8c28778-webhook-cert\") pod \"maas-controller-8494c84d99-nxrx7\" (UID: \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\") " pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:25:50.763213 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.763189    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th8ch\" (UniqueName: \"kubernetes.io/projected/f7a12aa8-7724-43c9-b8c8-5256d8c28778-kube-api-access-th8ch\") pod \"maas-controller-8494c84d99-nxrx7\" (UID: \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\") " pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:25:50.886971 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:50.886873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:25:51.023871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:51.023844    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8494c84d99-nxrx7"]
Jul 26 18:25:51.026331 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:25:51.026298    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a12aa8_7724_43c9_b8c8_5256d8c28778.slice/crio-262bce1fb60cd64371409fea27a63859960b457e81d339cf2d6267b139695b68 WatchSource:0}: Error finding container 262bce1fb60cd64371409fea27a63859960b457e81d339cf2d6267b139695b68: Status 404 returned error can't find the container with id 262bce1fb60cd64371409fea27a63859960b457e81d339cf2d6267b139695b68
Jul 26 18:25:51.080855 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:51.080820    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-nxrx7" event={"ID":"f7a12aa8-7724-43c9-b8c8-5256d8c28778","Type":"ContainerStarted","Data":"262bce1fb60cd64371409fea27a63859960b457e81d339cf2d6267b139695b68"}
Jul 26 18:25:52.087197 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:52.087154    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-nxrx7" event={"ID":"f7a12aa8-7724-43c9-b8c8-5256d8c28778","Type":"ContainerStarted","Data":"ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2"}
Jul 26 18:25:52.087604 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:52.087307    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:25:52.108083 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:25:52.108025    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8494c84d99-nxrx7" podStartSLOduration=1.708613618 podStartE2EDuration="2.108008669s" totalImagesPullingTime="399.395051ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:25:52.106169331 +0000 UTC m=+823.430691430" watchObservedRunningTime="2026-07-26 18:25:52.108008669 +0000 UTC m=+823.432530770"
Jul 26 18:26:03.094463 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:03.094431    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:26:07.710912 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:07.710875    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8494c84d99-nxrx7"]
Jul 26 18:26:07.711340 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:07.711103    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8494c84d99-nxrx7" podUID="f7a12aa8-7724-43c9-b8c8-5256d8c28778" containerName="manager" containerID="cri-o://ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2" gracePeriod=30
Jul 26 18:26:07.960082 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:07.960055    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:26:08.023776 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.023677    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7a12aa8-7724-43c9-b8c8-5256d8c28778-webhook-cert\") pod \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\" (UID: \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\") "
Jul 26 18:26:08.023932 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.023825    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-th8ch\" (UniqueName: \"kubernetes.io/projected/f7a12aa8-7724-43c9-b8c8-5256d8c28778-kube-api-access-th8ch\") pod \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\" (UID: \"f7a12aa8-7724-43c9-b8c8-5256d8c28778\") "
Jul 26 18:26:08.025861 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.025833    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7a12aa8-7724-43c9-b8c8-5256d8c28778-kube-api-access-th8ch" (OuterVolumeSpecName: "kube-api-access-th8ch") pod "f7a12aa8-7724-43c9-b8c8-5256d8c28778" (UID: "f7a12aa8-7724-43c9-b8c8-5256d8c28778"). InnerVolumeSpecName "kube-api-access-th8ch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:26:08.025970 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.025894    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7a12aa8-7724-43c9-b8c8-5256d8c28778-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f7a12aa8-7724-43c9-b8c8-5256d8c28778" (UID: "f7a12aa8-7724-43c9-b8c8-5256d8c28778"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:26:08.124961 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.124923    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th8ch\" (UniqueName: \"kubernetes.io/projected/f7a12aa8-7724-43c9-b8c8-5256d8c28778-kube-api-access-th8ch\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:26:08.124961 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.124953    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7a12aa8-7724-43c9-b8c8-5256d8c28778-webhook-cert\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:26:08.170289 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.170253    2559 generic.go:356] "Generic (PLEG): container finished" podID="f7a12aa8-7724-43c9-b8c8-5256d8c28778" containerID="ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2" exitCode=0
Jul 26 18:26:08.170461 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.170304    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-nxrx7"
Jul 26 18:26:08.170461 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.170300    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-nxrx7" event={"ID":"f7a12aa8-7724-43c9-b8c8-5256d8c28778","Type":"ContainerDied","Data":"ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2"}
Jul 26 18:26:08.170461 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.170435    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-nxrx7" event={"ID":"f7a12aa8-7724-43c9-b8c8-5256d8c28778","Type":"ContainerDied","Data":"262bce1fb60cd64371409fea27a63859960b457e81d339cf2d6267b139695b68"}
Jul 26 18:26:08.170565 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.170465    2559 scope.go:117] "RemoveContainer" containerID="ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2"
Jul 26 18:26:08.179552 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.179533    2559 scope.go:117] "RemoveContainer" containerID="ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2"
Jul 26 18:26:08.179834 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:26:08.179812    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2\": container with ID starting with ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2 not found: ID does not exist" containerID="ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2"
Jul 26 18:26:08.179887 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.179843    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2"} err="failed to get container status \"ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2\": rpc error: code = NotFound desc = could not find container \"ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2\": container with ID starting with ceaf5e1e1ec01063831e4d7d0daff0397e1d11b961ee277eb893f771d09c7fa2 not found: ID does not exist"
Jul 26 18:26:08.195542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.195513    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8494c84d99-nxrx7"]
Jul 26 18:26:08.199384 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.199354    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8494c84d99-nxrx7"]
Jul 26 18:26:08.954510 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.954471    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8494c84d99-k6zhp"]
Jul 26 18:26:08.954974 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.954954    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7a12aa8-7724-43c9-b8c8-5256d8c28778" containerName="manager"
Jul 26 18:26:08.954974 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.954969    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7a12aa8-7724-43c9-b8c8-5256d8c28778" containerName="manager"
Jul 26 18:26:08.955066 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.955033    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7a12aa8-7724-43c9-b8c8-5256d8c28778" containerName="manager"
Jul 26 18:26:08.959388 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.959367    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:26:08.962585 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.962563    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 18:26:08.963959 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.963940    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8srmb\""
Jul 26 18:26:08.964072 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.963964    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 18:26:08.970953 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:08.970925    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8494c84d99-k6zhp"]
Jul 26 18:26:09.033835 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.033788    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9dwb\" (UniqueName: \"kubernetes.io/projected/402e9e41-e678-4722-b6a2-13a0af69febe-kube-api-access-j9dwb\") pod \"maas-controller-8494c84d99-k6zhp\" (UID: \"402e9e41-e678-4722-b6a2-13a0af69febe\") " pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:26:09.034021 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.033840    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/402e9e41-e678-4722-b6a2-13a0af69febe-webhook-cert\") pod \"maas-controller-8494c84d99-k6zhp\" (UID: \"402e9e41-e678-4722-b6a2-13a0af69febe\") " pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:26:09.134981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.134931    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9dwb\" (UniqueName: \"kubernetes.io/projected/402e9e41-e678-4722-b6a2-13a0af69febe-kube-api-access-j9dwb\") pod \"maas-controller-8494c84d99-k6zhp\" (UID: \"402e9e41-e678-4722-b6a2-13a0af69febe\") " pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:26:09.134981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.134979    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/402e9e41-e678-4722-b6a2-13a0af69febe-webhook-cert\") pod \"maas-controller-8494c84d99-k6zhp\" (UID: \"402e9e41-e678-4722-b6a2-13a0af69febe\") " pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:26:09.138212 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.138182    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 26 18:26:09.144175 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.144151    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9dwb\" (UniqueName: \"kubernetes.io/projected/402e9e41-e678-4722-b6a2-13a0af69febe-kube-api-access-j9dwb\") pod \"maas-controller-8494c84d99-k6zhp\" (UID: \"402e9e41-e678-4722-b6a2-13a0af69febe\") " pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:26:09.147448 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.147422    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/402e9e41-e678-4722-b6a2-13a0af69febe-webhook-cert\") pod \"maas-controller-8494c84d99-k6zhp\" (UID: \"402e9e41-e678-4722-b6a2-13a0af69febe\") " pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:26:09.178305 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.178265    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7a12aa8-7724-43c9-b8c8-5256d8c28778" path="/var/lib/kubelet/pods/f7a12aa8-7724-43c9-b8c8-5256d8c28778/volumes"
Jul 26 18:26:09.273124 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.273033    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8srmb\""
Jul 26 18:26:09.279807 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.279775    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:26:09.417996 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.417968    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8494c84d99-k6zhp"]
Jul 26 18:26:09.419790 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:26:09.419764    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod402e9e41_e678_4722_b6a2_13a0af69febe.slice/crio-64b469b683bbad1795f6543c2c334402226ec7881325bb499608fb9201f40f15 WatchSource:0}: Error finding container 64b469b683bbad1795f6543c2c334402226ec7881325bb499608fb9201f40f15: Status 404 returned error can't find the container with id 64b469b683bbad1795f6543c2c334402226ec7881325bb499608fb9201f40f15
Jul 26 18:26:09.746699 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:09.746668    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 26 18:26:10.181485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:10.181453    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-k6zhp" event={"ID":"402e9e41-e678-4722-b6a2-13a0af69febe","Type":"ContainerStarted","Data":"5a1eb1816fc59261011aa1ed4f6680f493ab51793b08d7cf10a5af380f8cde30"}
Jul 26 18:26:10.181974 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:10.181514    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8494c84d99-k6zhp" event={"ID":"402e9e41-e678-4722-b6a2-13a0af69febe","Type":"ContainerStarted","Data":"64b469b683bbad1795f6543c2c334402226ec7881325bb499608fb9201f40f15"}
Jul 26 18:26:10.181974 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:10.181550    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:26:10.200373 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:10.200293    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8494c84d99-k6zhp" podStartSLOduration=1.878042716 podStartE2EDuration="2.200272738s" totalImagesPullingTime="322.230022ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:26:10.199468995 +0000 UTC m=+841.523991093" watchObservedRunningTime="2026-07-26 18:26:10.200272738 +0000 UTC m=+841.524794836"
Jul 26 18:26:21.189331 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:26:21.189303    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8494c84d99-k6zhp"
Jul 26 18:27:09.788055 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:27:09.787945    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:27:09.792527 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:27:09.792418    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:30:44.328872 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:44.328834    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"]
Jul 26 18:30:44.329428 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:44.329104    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9" podUID="40864ec8-6f22-446b-bf10-3ce81702368d" containerName="manager" containerID="cri-o://380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60" gracePeriod=10
Jul 26 18:30:47.281453 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.281429    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:30:47.414346 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.414322    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40864ec8-6f22-446b-bf10-3ce81702368d-extensions-socket-volume\") pod \"40864ec8-6f22-446b-bf10-3ce81702368d\" (UID: \"40864ec8-6f22-446b-bf10-3ce81702368d\") "
Jul 26 18:30:47.414491 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.414362    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twtdp\" (UniqueName: \"kubernetes.io/projected/40864ec8-6f22-446b-bf10-3ce81702368d-kube-api-access-twtdp\") pod \"40864ec8-6f22-446b-bf10-3ce81702368d\" (UID: \"40864ec8-6f22-446b-bf10-3ce81702368d\") "
Jul 26 18:30:47.414772 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.414744    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40864ec8-6f22-446b-bf10-3ce81702368d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "40864ec8-6f22-446b-bf10-3ce81702368d" (UID: "40864ec8-6f22-446b-bf10-3ce81702368d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:30:47.416329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.416313    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40864ec8-6f22-446b-bf10-3ce81702368d-kube-api-access-twtdp" (OuterVolumeSpecName: "kube-api-access-twtdp") pod "40864ec8-6f22-446b-bf10-3ce81702368d" (UID: "40864ec8-6f22-446b-bf10-3ce81702368d"). InnerVolumeSpecName "kube-api-access-twtdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:30:47.497912 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.497879    2559 generic.go:356] "Generic (PLEG): container finished" podID="40864ec8-6f22-446b-bf10-3ce81702368d" containerID="380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60" exitCode=0
Jul 26 18:30:47.498074 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.497943    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"
Jul 26 18:30:47.498074 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.497974    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9" event={"ID":"40864ec8-6f22-446b-bf10-3ce81702368d","Type":"ContainerDied","Data":"380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60"}
Jul 26 18:30:47.498074 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.498030    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9" event={"ID":"40864ec8-6f22-446b-bf10-3ce81702368d","Type":"ContainerDied","Data":"66c0c5559b4d9a7aafbaf3eea683e9648ddd15824b6808bc68c116f58903a96a"}
Jul 26 18:30:47.498074 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.498053    2559 scope.go:117] "RemoveContainer" containerID="380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60"
Jul 26 18:30:47.508992 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.508976    2559 scope.go:117] "RemoveContainer" containerID="380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60"
Jul 26 18:30:47.509279 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:30:47.509237    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60\": container with ID starting with 380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60 not found: ID does not exist" containerID="380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60"
Jul 26 18:30:47.509344 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.509270    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60"} err="failed to get container status \"380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60\": rpc error: code = NotFound desc = could not find container \"380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60\": container with ID starting with 380d54eb0eb05c0ab7c71119c707d0459c7ba030a216b3081e0ab3202c924e60 not found: ID does not exist"
Jul 26 18:30:47.516018 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.515996    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40864ec8-6f22-446b-bf10-3ce81702368d-extensions-socket-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:30:47.516102 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.516023    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twtdp\" (UniqueName: \"kubernetes.io/projected/40864ec8-6f22-446b-bf10-3ce81702368d-kube-api-access-twtdp\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:30:47.521673 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.521653    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"]
Jul 26 18:30:47.525178 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:47.525153    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29ft9"]
Jul 26 18:30:49.176035 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:30:49.176003    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40864ec8-6f22-446b-bf10-3ce81702368d" path="/var/lib/kubelet/pods/40864ec8-6f22-446b-bf10-3ce81702368d/volumes"
Jul 26 18:31:45.428441 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.428347    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"]
Jul 26 18:31:45.429043 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.428967    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40864ec8-6f22-446b-bf10-3ce81702368d" containerName="manager"
Jul 26 18:31:45.429043 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.428987    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="40864ec8-6f22-446b-bf10-3ce81702368d" containerName="manager"
Jul 26 18:31:45.429168 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.429114    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="40864ec8-6f22-446b-bf10-3ce81702368d" containerName="manager"
Jul 26 18:31:45.431950 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.431929    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:31:45.434865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.434847    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8rshk\""
Jul 26 18:31:45.442536 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.442512    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"]
Jul 26 18:31:45.547894 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.547862    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3606262-f9fa-4397-9d5c-dd4521f2c146-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-snpqf\" (UID: \"a3606262-f9fa-4397-9d5c-dd4521f2c146\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:31:45.548074 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.548030    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfgw2\" (UniqueName: \"kubernetes.io/projected/a3606262-f9fa-4397-9d5c-dd4521f2c146-kube-api-access-xfgw2\") pod \"kuadrant-operator-controller-manager-86878c769f-snpqf\" (UID: \"a3606262-f9fa-4397-9d5c-dd4521f2c146\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:31:45.648776 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.648736    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfgw2\" (UniqueName: \"kubernetes.io/projected/a3606262-f9fa-4397-9d5c-dd4521f2c146-kube-api-access-xfgw2\") pod \"kuadrant-operator-controller-manager-86878c769f-snpqf\" (UID: \"a3606262-f9fa-4397-9d5c-dd4521f2c146\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:31:45.648776 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.648781    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3606262-f9fa-4397-9d5c-dd4521f2c146-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-snpqf\" (UID: \"a3606262-f9fa-4397-9d5c-dd4521f2c146\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:31:45.649151 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.649133    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3606262-f9fa-4397-9d5c-dd4521f2c146-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-snpqf\" (UID: \"a3606262-f9fa-4397-9d5c-dd4521f2c146\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:31:45.659586 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.659556    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfgw2\" (UniqueName: \"kubernetes.io/projected/a3606262-f9fa-4397-9d5c-dd4521f2c146-kube-api-access-xfgw2\") pod \"kuadrant-operator-controller-manager-86878c769f-snpqf\" (UID: \"a3606262-f9fa-4397-9d5c-dd4521f2c146\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:31:45.743960 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.743878    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:31:45.883412 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:45.883387    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"]
Jul 26 18:31:45.885753 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:31:45.885720    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3606262_f9fa_4397_9d5c_dd4521f2c146.slice/crio-1a77a690aaa85d8f7e0c22a532e7513986b43e3702354b6c7436797f69719726 WatchSource:0}: Error finding container 1a77a690aaa85d8f7e0c22a532e7513986b43e3702354b6c7436797f69719726: Status 404 returned error can't find the container with id 1a77a690aaa85d8f7e0c22a532e7513986b43e3702354b6c7436797f69719726
Jul 26 18:31:46.776882 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:46.776846    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf" event={"ID":"a3606262-f9fa-4397-9d5c-dd4521f2c146","Type":"ContainerStarted","Data":"bfe53faad1af3ad57fa42beb3213fc9d4877b40203be57454e5f4249e96f20c8"}
Jul 26 18:31:46.777317 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:46.776898    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf" event={"ID":"a3606262-f9fa-4397-9d5c-dd4521f2c146","Type":"ContainerStarted","Data":"1a77a690aaa85d8f7e0c22a532e7513986b43e3702354b6c7436797f69719726"}
Jul 26 18:31:46.777317 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:46.776949    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:31:46.804207 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:46.804148    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf" podStartSLOduration=1.804127646 podStartE2EDuration="1.804127646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:31:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:31:46.802293715 +0000 UTC m=+1178.126815812" watchObservedRunningTime="2026-07-26 18:31:46.804127646 +0000 UTC m=+1178.128649744"
Jul 26 18:31:57.783587 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:31:57.783558    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-snpqf"
Jul 26 18:32:09.819696 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:32:09.819565    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:32:09.824517 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:32:09.824498    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:34:41.195005 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.194925    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"]
Jul 26 18:34:41.198689 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.198668    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.201342 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.201323    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-d30d1f16-kserve-self-signed-certs\""
Jul 26 18:34:41.209633 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.209591    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"]
Jul 26 18:34:41.262427 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.262403    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-dshm\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.262554 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.262432    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-model-cache\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.262554 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.262452    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tls-certs\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.262554 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.262469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-home\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.262696 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.262566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tmp-dir\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.262696 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.262606    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fss2x\" (UniqueName: \"kubernetes.io/projected/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-kube-api-access-fss2x\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.363611 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.363579    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-dshm\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.363765 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.363632    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-model-cache\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.363765 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.363660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tls-certs\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.363765 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.363679    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-home\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.363765 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.363709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tmp-dir\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.363765 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.363748    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fss2x\" (UniqueName: \"kubernetes.io/projected/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-kube-api-access-fss2x\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.364122 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.364102    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-model-cache\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.364184 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.364153    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tmp-dir\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.364238 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.364194    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-home\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.366038 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.366015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-dshm\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.366291 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.366270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tls-certs\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.372008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.371984    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fss2x\" (UniqueName: \"kubernetes.io/projected/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-kube-api-access-fss2x\") pod \"e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") " pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.508224 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.508143    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:41.640043 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:41.640019    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"]
Jul 26 18:34:41.641376 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:34:41.641347    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6aa0e3ea_7bc7_4a21_aac4_67b0a5f8e87a.slice/crio-1cf4b7a3bf8b2dd0a1e800394a92fc3e765c42731eb4c5cd2acd90018f7c8362 WatchSource:0}: Error finding container 1cf4b7a3bf8b2dd0a1e800394a92fc3e765c42731eb4c5cd2acd90018f7c8362: Status 404 returned error can't find the container with id 1cf4b7a3bf8b2dd0a1e800394a92fc3e765c42731eb4c5cd2acd90018f7c8362
Jul 26 18:34:42.620688 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:42.620655    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt" event={"ID":"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a","Type":"ContainerStarted","Data":"d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231"}
Jul 26 18:34:42.621063 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:42.620695    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt" event={"ID":"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a","Type":"ContainerStarted","Data":"1cf4b7a3bf8b2dd0a1e800394a92fc3e765c42731eb4c5cd2acd90018f7c8362"}
Jul 26 18:34:42.639340 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:42.639290    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt" podStartSLOduration=1.639276025 podStartE2EDuration="1.639276025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:34:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:34:42.637699964 +0000 UTC m=+1353.962222062" watchObservedRunningTime="2026-07-26 18:34:42.639276025 +0000 UTC m=+1353.963798122"
Jul 26 18:34:51.509271 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:51.509226    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:51.509768 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:51.509288    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:51.521987 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:51.521961    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:34:51.676266 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:34:51.676238    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:35:18.322379 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:18.322343    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"]
Jul 26 18:35:33.345636 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.345550    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt" podUID="6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" containerName="main" containerID="cri-o://d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231" gracePeriod=45
Jul 26 18:35:33.378669 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:35:33.378640    2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6aa0e3ea_7bc7_4a21_aac4_67b0a5f8e87a.slice/crio-d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231.scope\": RecentStats: unable to find data in memory cache]"
Jul 26 18:35:33.586840 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.586812    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:35:33.736885 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.736853    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-home\") pod \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") "
Jul 26 18:35:33.737083 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.736936    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-dshm\") pod \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") "
Jul 26 18:35:33.737083 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.736970    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tls-certs\") pod \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") "
Jul 26 18:35:33.737083 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.736998    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-model-cache\") pod \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") "
Jul 26 18:35:33.737083 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.737058    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fss2x\" (UniqueName: \"kubernetes.io/projected/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-kube-api-access-fss2x\") pod \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") "
Jul 26 18:35:33.737311 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.737158    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tmp-dir\") pod \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\" (UID: \"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a\") "
Jul 26 18:35:33.737371 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.737344    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-model-cache" (OuterVolumeSpecName: "model-cache") pod "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" (UID: "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:33.737473 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.737174    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-home" (OuterVolumeSpecName: "home") pod "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" (UID: "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:33.737592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.737493    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:35:33.737592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.737574    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" (UID: "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:33.739232 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.739210    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-dshm" (OuterVolumeSpecName: "dshm") pod "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" (UID: "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:35:33.739397 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.739373    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-kube-api-access-fss2x" (OuterVolumeSpecName: "kube-api-access-fss2x") pod "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" (UID: "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a"). InnerVolumeSpecName "kube-api-access-fss2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:35:33.739502 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.739483    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" (UID: "6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:35:33.838992 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.838961    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:35:33.838992 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.838987    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:35:33.838992 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.838999    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fss2x\" (UniqueName: \"kubernetes.io/projected/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-kube-api-access-fss2x\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:35:33.839183 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.839008    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:35:33.839183 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.839016    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:35:33.870543 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.870513    2559 generic.go:356] "Generic (PLEG): container finished" podID="6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" containerID="d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231" exitCode=0
Jul 26 18:35:33.870719 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.870557    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"
Jul 26 18:35:33.870719 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.870600    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt" event={"ID":"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a","Type":"ContainerDied","Data":"d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231"}
Jul 26 18:35:33.870719 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.870657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt" event={"ID":"6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a","Type":"ContainerDied","Data":"1cf4b7a3bf8b2dd0a1e800394a92fc3e765c42731eb4c5cd2acd90018f7c8362"}
Jul 26 18:35:33.870719 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.870680    2559 scope.go:117] "RemoveContainer" containerID="d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231"
Jul 26 18:35:33.879848 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.879828    2559 scope.go:117] "RemoveContainer" containerID="d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231"
Jul 26 18:35:33.880106 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:35:33.880090    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231\": container with ID starting with d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231 not found: ID does not exist" containerID="d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231"
Jul 26 18:35:33.880151 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.880116    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231"} err="failed to get container status \"d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231\": rpc error: code = NotFound desc = could not find container \"d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231\": container with ID starting with d777db14be2efbe6f0a6068972de4825d96f6ef3bc8143e63e6f0fa0d26e6231 not found: ID does not exist"
Jul 26 18:35:33.891559 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.891536    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"]
Jul 26 18:35:33.895037 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:33.895015    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-d30d1f16-kserve-6557dc5b79-wdhjt"]
Jul 26 18:35:35.178764 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:35:35.178730    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" path="/var/lib/kubelet/pods/6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a/volumes"
Jul 26 18:37:09.858759 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:37:09.858648    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:37:09.864276 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:37:09.864255    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:38:57.123210 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.123174    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"]
Jul 26 18:38:57.123801 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.123629    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" containerName="main"
Jul 26 18:38:57.123801 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.123645    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" containerName="main"
Jul 26 18:38:57.123801 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.123722    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aa0e3ea-7bc7-4a21-aac4-67b0a5f8e87a" containerName="main"
Jul 26 18:38:57.127087 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.127072    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:38:57.130751 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.130727    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-67813b6a\""
Jul 26 18:38:57.130852 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.130735    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-67813b6a-dockercfg-rf8t9\""
Jul 26 18:38:57.138554 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.138534    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"]
Jul 26 18:38:57.321503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.321471    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-plugins-config-volume\") pod \"payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f\" (UID: \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\") " pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:38:57.321704 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.321631    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfmnl\" (UniqueName: \"kubernetes.io/projected/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-kube-api-access-zfmnl\") pod \"payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f\" (UID: \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\") " pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:38:57.423035 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.423002    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfmnl\" (UniqueName: \"kubernetes.io/projected/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-kube-api-access-zfmnl\") pod \"payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f\" (UID: \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\") " pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:38:57.423203 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.423068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-plugins-config-volume\") pod \"payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f\" (UID: \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\") " pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:38:57.423765 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.423747    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-plugins-config-volume\") pod \"payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f\" (UID: \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\") " pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:38:57.432253 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.432228    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfmnl\" (UniqueName: \"kubernetes.io/projected/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-kube-api-access-zfmnl\") pod \"payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f\" (UID: \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\") " pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:38:57.438497 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.438474    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:38:57.580359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.580317    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"]
Jul 26 18:38:57.584547 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:38:57.584520    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b79a3b_0ac1_4ebb_8e60_e4761e5678bb.slice/crio-cdd75130a8e1dd25d23c569e1d261b54703546eff3ae06d228c90094940b83bd WatchSource:0}: Error finding container cdd75130a8e1dd25d23c569e1d261b54703546eff3ae06d228c90094940b83bd: Status 404 returned error can't find the container with id cdd75130a8e1dd25d23c569e1d261b54703546eff3ae06d228c90094940b83bd
Jul 26 18:38:57.586531 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.586505    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:38:57.837241 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:57.837153    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f" event={"ID":"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb","Type":"ContainerStarted","Data":"cdd75130a8e1dd25d23c569e1d261b54703546eff3ae06d228c90094940b83bd"}
Jul 26 18:38:59.851181 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:59.851145    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f" event={"ID":"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb","Type":"ContainerStarted","Data":"5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d"}
Jul 26 18:38:59.851670 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:59.851210    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:38:59.894723 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:38:59.894671    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f" podStartSLOduration=1.464826953 podStartE2EDuration="2.894654983s" totalImagesPullingTime="1.42982803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:38:59.892099741 +0000 UTC m=+1611.216621831" watchObservedRunningTime="2026-07-26 18:38:59.894654983 +0000 UTC m=+1611.219177081"
Jul 26 18:39:10.858985 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:10.858957    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:39:32.179576 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:32.179543    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"]
Jul 26 18:39:32.180106 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:32.179821    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f" podUID="c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb" containerName="payload-processing" containerID="cri-o://5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d" gracePeriod=30
Jul 26 18:39:32.431184 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:32.431125    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:39:32.568125 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:32.568096    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-plugins-config-volume\") pod \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\" (UID: \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\") "
Jul 26 18:39:32.568289 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:32.568147    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfmnl\" (UniqueName: \"kubernetes.io/projected/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-kube-api-access-zfmnl\") pod \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\" (UID: \"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb\") "
Jul 26 18:39:32.568507 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:32.568482    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb" (UID: "c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:39:32.570414 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:32.570390    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-kube-api-access-zfmnl" (OuterVolumeSpecName: "kube-api-access-zfmnl") pod "c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb" (UID: "c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb"). InnerVolumeSpecName "kube-api-access-zfmnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:39:32.669354 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:32.669319    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:39:32.669354 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:32.669348    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfmnl\" (UniqueName: \"kubernetes.io/projected/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb-kube-api-access-zfmnl\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:39:33.013326 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.013290    2559 generic.go:356] "Generic (PLEG): container finished" podID="c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb" containerID="5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d" exitCode=0
Jul 26 18:39:33.013524 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.013344    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"
Jul 26 18:39:33.013524 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.013365    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f" event={"ID":"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb","Type":"ContainerDied","Data":"5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d"}
Jul 26 18:39:33.013524 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.013410    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f" event={"ID":"c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb","Type":"ContainerDied","Data":"cdd75130a8e1dd25d23c569e1d261b54703546eff3ae06d228c90094940b83bd"}
Jul 26 18:39:33.013524 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.013433    2559 scope.go:117] "RemoveContainer" containerID="5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d"
Jul 26 18:39:33.022448 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.022421    2559 scope.go:117] "RemoveContainer" containerID="5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d"
Jul 26 18:39:33.022782 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:39:33.022756    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d\": container with ID starting with 5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d not found: ID does not exist" containerID="5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d"
Jul 26 18:39:33.022867 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.022789    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d"} err="failed to get container status \"5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d\": rpc error: code = NotFound desc = could not find container \"5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d\": container with ID starting with 5182ed770f10152b5705d5a8208130f928f40454f4c039bce61f784eda228f4d not found: ID does not exist"
Jul 26 18:39:33.048542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.048517    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"]
Jul 26 18:39:33.052757 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.052731    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-67813b6a-857c49fd8d-8hs9f"]
Jul 26 18:39:33.176353 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:33.176321    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb" path="/var/lib/kubelet/pods/c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb/volumes"
Jul 26 18:39:43.947015 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.946981    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"]
Jul 26 18:39:43.947463 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.947389    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb" containerName="payload-processing"
Jul 26 18:39:43.947463 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.947402    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb" containerName="payload-processing"
Jul 26 18:39:43.947533 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.947467    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5b79a3b-0ac1-4ebb-8e60-e4761e5678bb" containerName="payload-processing"
Jul 26 18:39:43.950453 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.950433    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:39:43.953985 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.953966    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-94b43cc4\""
Jul 26 18:39:43.955359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.955339    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-94b43cc4-dockercfg-k8cn4\""
Jul 26 18:39:43.965078 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.965055    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"]
Jul 26 18:39:43.989128 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.989103    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"]
Jul 26 18:39:43.992595 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:43.992575    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:44.003840 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.003820    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"]
Jul 26 18:39:44.070309 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.070260    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d11f78b-287f-41b1-8873-3440f58e503b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp\" (UID: \"0d11f78b-287f-41b1-8873-3440f58e503b\") " pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:39:44.070309 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.070308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flxqp\" (UniqueName: \"kubernetes.io/projected/0d11f78b-287f-41b1-8873-3440f58e503b-kube-api-access-flxqp\") pod \"payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp\" (UID: \"0d11f78b-287f-41b1-8873-3440f58e503b\") " pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:39:44.070548 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.070381    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5df0dbf-6336-4a06-a3d5-56f634d00487-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh\" (UID: \"e5df0dbf-6336-4a06-a3d5-56f634d00487\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:44.070548 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.070412    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j942h\" (UniqueName: \"kubernetes.io/projected/e5df0dbf-6336-4a06-a3d5-56f634d00487-kube-api-access-j942h\") pod \"payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh\" (UID: \"e5df0dbf-6336-4a06-a3d5-56f634d00487\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:44.171004 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.170971    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d11f78b-287f-41b1-8873-3440f58e503b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp\" (UID: \"0d11f78b-287f-41b1-8873-3440f58e503b\") " pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:39:44.171204 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.171008    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flxqp\" (UniqueName: \"kubernetes.io/projected/0d11f78b-287f-41b1-8873-3440f58e503b-kube-api-access-flxqp\") pod \"payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp\" (UID: \"0d11f78b-287f-41b1-8873-3440f58e503b\") " pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:39:44.171204 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.171104    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5df0dbf-6336-4a06-a3d5-56f634d00487-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh\" (UID: \"e5df0dbf-6336-4a06-a3d5-56f634d00487\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:44.171204 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.171158    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j942h\" (UniqueName: \"kubernetes.io/projected/e5df0dbf-6336-4a06-a3d5-56f634d00487-kube-api-access-j942h\") pod \"payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh\" (UID: \"e5df0dbf-6336-4a06-a3d5-56f634d00487\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:44.171803 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.171784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d11f78b-287f-41b1-8873-3440f58e503b-plugins-config-volume\") pod \"payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp\" (UID: \"0d11f78b-287f-41b1-8873-3440f58e503b\") " pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:39:44.171872 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.171856    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5df0dbf-6336-4a06-a3d5-56f634d00487-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh\" (UID: \"e5df0dbf-6336-4a06-a3d5-56f634d00487\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:44.179993 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.179964    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flxqp\" (UniqueName: \"kubernetes.io/projected/0d11f78b-287f-41b1-8873-3440f58e503b-kube-api-access-flxqp\") pod \"payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp\" (UID: \"0d11f78b-287f-41b1-8873-3440f58e503b\") " pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:39:44.180093 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.180015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j942h\" (UniqueName: \"kubernetes.io/projected/e5df0dbf-6336-4a06-a3d5-56f634d00487-kube-api-access-j942h\") pod \"payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh\" (UID: \"e5df0dbf-6336-4a06-a3d5-56f634d00487\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:44.260502 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.260432    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:39:44.302776 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.302742    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:44.408985 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.408721    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"]
Jul 26 18:39:44.411980 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:39:44.411947    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d11f78b_287f_41b1_8873_3440f58e503b.slice/crio-0bd76acf74c8ac4a9501cb158214aebe489d043c4ac9a2ad0e8c5764c2605427 WatchSource:0}: Error finding container 0bd76acf74c8ac4a9501cb158214aebe489d043c4ac9a2ad0e8c5764c2605427: Status 404 returned error can't find the container with id 0bd76acf74c8ac4a9501cb158214aebe489d043c4ac9a2ad0e8c5764c2605427
Jul 26 18:39:44.461008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:44.460986    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"]
Jul 26 18:39:44.463183 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:39:44.463151    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5df0dbf_6336_4a06_a3d5_56f634d00487.slice/crio-e9d3fd959df1b25f1f591ea01f29ca5a0e49e6b441dc46ee6e6529fcaa7e2873 WatchSource:0}: Error finding container e9d3fd959df1b25f1f591ea01f29ca5a0e49e6b441dc46ee6e6529fcaa7e2873: Status 404 returned error can't find the container with id e9d3fd959df1b25f1f591ea01f29ca5a0e49e6b441dc46ee6e6529fcaa7e2873
Jul 26 18:39:45.075558 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:45.075518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp" event={"ID":"0d11f78b-287f-41b1-8873-3440f58e503b","Type":"ContainerStarted","Data":"a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4"}
Jul 26 18:39:45.076037 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:45.075589    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp" event={"ID":"0d11f78b-287f-41b1-8873-3440f58e503b","Type":"ContainerStarted","Data":"0bd76acf74c8ac4a9501cb158214aebe489d043c4ac9a2ad0e8c5764c2605427"}
Jul 26 18:39:45.076037 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:45.075641    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:39:45.077036 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:45.077013    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh" event={"ID":"e5df0dbf-6336-4a06-a3d5-56f634d00487","Type":"ContainerStarted","Data":"8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322"}
Jul 26 18:39:45.077172 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:45.077058    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh" event={"ID":"e5df0dbf-6336-4a06-a3d5-56f634d00487","Type":"ContainerStarted","Data":"e9d3fd959df1b25f1f591ea01f29ca5a0e49e6b441dc46ee6e6529fcaa7e2873"}
Jul 26 18:39:45.077172 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:45.077089    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:45.094629 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:45.094566    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp" podStartSLOduration=2.094551085 podStartE2EDuration="2.094551085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:39:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:39:45.092463891 +0000 UTC m=+1656.416985989" watchObservedRunningTime="2026-07-26 18:39:45.094551085 +0000 UTC m=+1656.419073231"
Jul 26 18:39:45.109303 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:45.109252    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh" podStartSLOduration=2.109237544 podStartE2EDuration="2.109237544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:39:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:39:45.108214563 +0000 UTC m=+1656.432736661" watchObservedRunningTime="2026-07-26 18:39:45.109237544 +0000 UTC m=+1656.433759641"
Jul 26 18:39:56.083794 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:56.083766    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:39:56.084209 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:39:56.083837    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:40:18.661828 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:18.661787    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"]
Jul 26 18:40:18.662398 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:18.662312    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp" podUID="0d11f78b-287f-41b1-8873-3440f58e503b" containerName="payload-processing" containerID="cri-o://a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4" gracePeriod=30
Jul 26 18:40:18.672574 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:18.672549    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"]
Jul 26 18:40:18.672844 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:18.672817    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh" podUID="e5df0dbf-6336-4a06-a3d5-56f634d00487" containerName="payload-pre-processing" containerID="cri-o://8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322" gracePeriod=30
Jul 26 18:40:18.944875 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:18.944848    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:40:18.948702 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:18.948681    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:40:19.000287 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.000253    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-flxqp\" (UniqueName: \"kubernetes.io/projected/0d11f78b-287f-41b1-8873-3440f58e503b-kube-api-access-flxqp\") pod \"0d11f78b-287f-41b1-8873-3440f58e503b\" (UID: \"0d11f78b-287f-41b1-8873-3440f58e503b\") "
Jul 26 18:40:19.000459 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.000316    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5df0dbf-6336-4a06-a3d5-56f634d00487-plugins-config-volume\") pod \"e5df0dbf-6336-4a06-a3d5-56f634d00487\" (UID: \"e5df0dbf-6336-4a06-a3d5-56f634d00487\") "
Jul 26 18:40:19.000459 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.000360    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j942h\" (UniqueName: \"kubernetes.io/projected/e5df0dbf-6336-4a06-a3d5-56f634d00487-kube-api-access-j942h\") pod \"e5df0dbf-6336-4a06-a3d5-56f634d00487\" (UID: \"e5df0dbf-6336-4a06-a3d5-56f634d00487\") "
Jul 26 18:40:19.000794 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.000764    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d11f78b-287f-41b1-8873-3440f58e503b-plugins-config-volume\") pod \"0d11f78b-287f-41b1-8873-3440f58e503b\" (UID: \"0d11f78b-287f-41b1-8873-3440f58e503b\") "
Jul 26 18:40:19.001037 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.000760    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5df0dbf-6336-4a06-a3d5-56f634d00487-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5df0dbf-6336-4a06-a3d5-56f634d00487" (UID: "e5df0dbf-6336-4a06-a3d5-56f634d00487"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:40:19.001185 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.001159    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d11f78b-287f-41b1-8873-3440f58e503b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0d11f78b-287f-41b1-8873-3440f58e503b" (UID: "0d11f78b-287f-41b1-8873-3440f58e503b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:40:19.001250 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.001187    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5df0dbf-6336-4a06-a3d5-56f634d00487-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:40:19.002579 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.002554    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d11f78b-287f-41b1-8873-3440f58e503b-kube-api-access-flxqp" (OuterVolumeSpecName: "kube-api-access-flxqp") pod "0d11f78b-287f-41b1-8873-3440f58e503b" (UID: "0d11f78b-287f-41b1-8873-3440f58e503b"). InnerVolumeSpecName "kube-api-access-flxqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:40:19.002725 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.002581    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5df0dbf-6336-4a06-a3d5-56f634d00487-kube-api-access-j942h" (OuterVolumeSpecName: "kube-api-access-j942h") pod "e5df0dbf-6336-4a06-a3d5-56f634d00487" (UID: "e5df0dbf-6336-4a06-a3d5-56f634d00487"). InnerVolumeSpecName "kube-api-access-j942h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:40:19.102118 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.102087    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flxqp\" (UniqueName: \"kubernetes.io/projected/0d11f78b-287f-41b1-8873-3440f58e503b-kube-api-access-flxqp\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:40:19.102118 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.102111    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j942h\" (UniqueName: \"kubernetes.io/projected/e5df0dbf-6336-4a06-a3d5-56f634d00487-kube-api-access-j942h\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:40:19.102118 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.102120    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d11f78b-287f-41b1-8873-3440f58e503b-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:40:19.244039 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.244002    2559 generic.go:356] "Generic (PLEG): container finished" podID="e5df0dbf-6336-4a06-a3d5-56f634d00487" containerID="8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322" exitCode=0
Jul 26 18:40:19.244206 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.244064    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"
Jul 26 18:40:19.244206 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.244097    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh" event={"ID":"e5df0dbf-6336-4a06-a3d5-56f634d00487","Type":"ContainerDied","Data":"8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322"}
Jul 26 18:40:19.244206 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.244166    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh" event={"ID":"e5df0dbf-6336-4a06-a3d5-56f634d00487","Type":"ContainerDied","Data":"e9d3fd959df1b25f1f591ea01f29ca5a0e49e6b441dc46ee6e6529fcaa7e2873"}
Jul 26 18:40:19.244206 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.244200    2559 scope.go:117] "RemoveContainer" containerID="8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322"
Jul 26 18:40:19.245500 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.245471    2559 generic.go:356] "Generic (PLEG): container finished" podID="0d11f78b-287f-41b1-8873-3440f58e503b" containerID="a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4" exitCode=0
Jul 26 18:40:19.245603 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.245532    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp" event={"ID":"0d11f78b-287f-41b1-8873-3440f58e503b","Type":"ContainerDied","Data":"a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4"}
Jul 26 18:40:19.245603 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.245561    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"
Jul 26 18:40:19.245603 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.245575    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp" event={"ID":"0d11f78b-287f-41b1-8873-3440f58e503b","Type":"ContainerDied","Data":"0bd76acf74c8ac4a9501cb158214aebe489d043c4ac9a2ad0e8c5764c2605427"}
Jul 26 18:40:19.253230 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.253215    2559 scope.go:117] "RemoveContainer" containerID="8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322"
Jul 26 18:40:19.253479 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:40:19.253455    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322\": container with ID starting with 8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322 not found: ID does not exist" containerID="8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322"
Jul 26 18:40:19.253523 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.253491    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322"} err="failed to get container status \"8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322\": rpc error: code = NotFound desc = could not find container \"8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322\": container with ID starting with 8fbd3ffb4e78050034356da683156b4f3a8a1dd177b46797d368980229cd6322 not found: ID does not exist"
Jul 26 18:40:19.253523 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.253509    2559 scope.go:117] "RemoveContainer" containerID="a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4"
Jul 26 18:40:19.260923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.260894    2559 scope.go:117] "RemoveContainer" containerID="a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4"
Jul 26 18:40:19.261351 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:40:19.261294    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4\": container with ID starting with a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4 not found: ID does not exist" containerID="a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4"
Jul 26 18:40:19.261351 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.261329    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4"} err="failed to get container status \"a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4\": rpc error: code = NotFound desc = could not find container \"a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4\": container with ID starting with a4f90f7a603e012cff98ea8c7a189a8d2e97c19d6a5c9750b2763751bc1794a4 not found: ID does not exist"
Jul 26 18:40:19.263631 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.263599    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"]
Jul 26 18:40:19.272480 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.272459    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-94b43cc4-9c94df4d4-xhplh"]
Jul 26 18:40:19.284770 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.284748    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"]
Jul 26 18:40:19.293032 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:19.293003    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-94b43cc4-59bbb78bbf-8g5mp"]
Jul 26 18:40:21.176484 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:21.176449    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d11f78b-287f-41b1-8873-3440f58e503b" path="/var/lib/kubelet/pods/0d11f78b-287f-41b1-8873-3440f58e503b/volumes"
Jul 26 18:40:21.176896 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:21.176834    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5df0dbf-6336-4a06-a3d5-56f634d00487" path="/var/lib/kubelet/pods/e5df0dbf-6336-4a06-a3d5-56f634d00487/volumes"
Jul 26 18:40:27.162931 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.162899    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"]
Jul 26 18:40:27.163372 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.163354    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d11f78b-287f-41b1-8873-3440f58e503b" containerName="payload-processing"
Jul 26 18:40:27.163438 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.163374    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d11f78b-287f-41b1-8873-3440f58e503b" containerName="payload-processing"
Jul 26 18:40:27.163438 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.163401    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5df0dbf-6336-4a06-a3d5-56f634d00487" containerName="payload-pre-processing"
Jul 26 18:40:27.163438 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.163407    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5df0dbf-6336-4a06-a3d5-56f634d00487" containerName="payload-pre-processing"
Jul 26 18:40:27.163551 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.163480    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d11f78b-287f-41b1-8873-3440f58e503b" containerName="payload-processing"
Jul 26 18:40:27.163551 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.163488    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5df0dbf-6336-4a06-a3d5-56f634d00487" containerName="payload-pre-processing"
Jul 26 18:40:27.166799 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.166785    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:40:27.170086 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.169638    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-48526096\""
Jul 26 18:40:27.170086 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.169912    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-48526096-dockercfg-7k96t\""
Jul 26 18:40:27.182611 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.182587    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"]
Jul 26 18:40:27.197917 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.197855    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"]
Jul 26 18:40:27.201291 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.201274    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:27.212841 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.212821    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"]
Jul 26 18:40:27.274902 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.274868    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlvt5\" (UniqueName: \"kubernetes.io/projected/55fd0d33-c2c5-4200-b4dc-94e98323f22d-kube-api-access-xlvt5\") pod \"payload-processing-e2e-ait-48526096-6778879b64-bfmb9\" (UID: \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\") " pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:27.275020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.274914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55fd0d33-c2c5-4200-b4dc-94e98323f22d-plugins-config-volume\") pod \"payload-processing-e2e-ait-48526096-6778879b64-bfmb9\" (UID: \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\") " pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:27.275020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.274943    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4zvd\" (UniqueName: \"kubernetes.io/projected/37645b37-2732-4974-944a-98feb90310ae-kube-api-access-b4zvd\") pod \"payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v\" (UID: \"37645b37-2732-4974-944a-98feb90310ae\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:40:27.275020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.274979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37645b37-2732-4974-944a-98feb90310ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v\" (UID: \"37645b37-2732-4974-944a-98feb90310ae\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:40:27.376194 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.376154    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37645b37-2732-4974-944a-98feb90310ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v\" (UID: \"37645b37-2732-4974-944a-98feb90310ae\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:40:27.376371 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.376249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlvt5\" (UniqueName: \"kubernetes.io/projected/55fd0d33-c2c5-4200-b4dc-94e98323f22d-kube-api-access-xlvt5\") pod \"payload-processing-e2e-ait-48526096-6778879b64-bfmb9\" (UID: \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\") " pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:27.376371 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.376276    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55fd0d33-c2c5-4200-b4dc-94e98323f22d-plugins-config-volume\") pod \"payload-processing-e2e-ait-48526096-6778879b64-bfmb9\" (UID: \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\") " pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:27.376371 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.376307    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4zvd\" (UniqueName: \"kubernetes.io/projected/37645b37-2732-4974-944a-98feb90310ae-kube-api-access-b4zvd\") pod \"payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v\" (UID: \"37645b37-2732-4974-944a-98feb90310ae\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:40:27.376893 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.376867    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37645b37-2732-4974-944a-98feb90310ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v\" (UID: \"37645b37-2732-4974-944a-98feb90310ae\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:40:27.377050 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.377030    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55fd0d33-c2c5-4200-b4dc-94e98323f22d-plugins-config-volume\") pod \"payload-processing-e2e-ait-48526096-6778879b64-bfmb9\" (UID: \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\") " pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:27.384924 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.384902    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4zvd\" (UniqueName: \"kubernetes.io/projected/37645b37-2732-4974-944a-98feb90310ae-kube-api-access-b4zvd\") pod \"payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v\" (UID: \"37645b37-2732-4974-944a-98feb90310ae\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:40:27.385064 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.385041    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlvt5\" (UniqueName: \"kubernetes.io/projected/55fd0d33-c2c5-4200-b4dc-94e98323f22d-kube-api-access-xlvt5\") pod \"payload-processing-e2e-ait-48526096-6778879b64-bfmb9\" (UID: \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\") " pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:27.478698 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.478585    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:40:27.512726 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.512697    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:27.627240 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.626952    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"]
Jul 26 18:40:27.635914 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:40:27.635882    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37645b37_2732_4974_944a_98feb90310ae.slice/crio-5b99409445064d8523d6609ed7595c26622b6e7d44684c9e5642a2c0ba9b2f17 WatchSource:0}: Error finding container 5b99409445064d8523d6609ed7595c26622b6e7d44684c9e5642a2c0ba9b2f17: Status 404 returned error can't find the container with id 5b99409445064d8523d6609ed7595c26622b6e7d44684c9e5642a2c0ba9b2f17
Jul 26 18:40:27.662170 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:27.662125    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"]
Jul 26 18:40:27.664671 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:40:27.664644    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55fd0d33_c2c5_4200_b4dc_94e98323f22d.slice/crio-3c8164370e4e3b17c255743f6753b9112661b44251f0cb3e66c72e23a775591f WatchSource:0}: Error finding container 3c8164370e4e3b17c255743f6753b9112661b44251f0cb3e66c72e23a775591f: Status 404 returned error can't find the container with id 3c8164370e4e3b17c255743f6753b9112661b44251f0cb3e66c72e23a775591f
Jul 26 18:40:28.292424 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:28.292384    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9" event={"ID":"55fd0d33-c2c5-4200-b4dc-94e98323f22d","Type":"ContainerStarted","Data":"78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6"}
Jul 26 18:40:28.292876 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:28.292454    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:28.292876 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:28.292481    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9" event={"ID":"55fd0d33-c2c5-4200-b4dc-94e98323f22d","Type":"ContainerStarted","Data":"3c8164370e4e3b17c255743f6753b9112661b44251f0cb3e66c72e23a775591f"}
Jul 26 18:40:28.293912 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:28.293889    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v" event={"ID":"37645b37-2732-4974-944a-98feb90310ae","Type":"ContainerStarted","Data":"a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55"}
Jul 26 18:40:28.294001 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:28.293936    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v" event={"ID":"37645b37-2732-4974-944a-98feb90310ae","Type":"ContainerStarted","Data":"5b99409445064d8523d6609ed7595c26622b6e7d44684c9e5642a2c0ba9b2f17"}
Jul 26 18:40:28.294038 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:28.294018    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:40:28.311013 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:28.310970    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9" podStartSLOduration=1.310956308 podStartE2EDuration="1.310956308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:40:28.309384326 +0000 UTC m=+1699.633906433" watchObservedRunningTime="2026-07-26 18:40:28.310956308 +0000 UTC m=+1699.635478405"
Jul 26 18:40:28.329029 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:28.328988    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v" podStartSLOduration=1.328974312 podStartE2EDuration="1.328974312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:40:28.327934351 +0000 UTC m=+1699.652456448" watchObservedRunningTime="2026-07-26 18:40:28.328974312 +0000 UTC m=+1699.653496409"
Jul 26 18:40:39.300658 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:39.300574    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:40:39.301026 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:40:39.300659    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:41:02.500325 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.500290    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"]
Jul 26 18:41:02.500841 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.500556    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9" podUID="55fd0d33-c2c5-4200-b4dc-94e98323f22d" containerName="payload-processing" containerID="cri-o://78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6" gracePeriod=30
Jul 26 18:41:02.507204 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.507175    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"]
Jul 26 18:41:02.507908 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.507654    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v" podUID="37645b37-2732-4974-944a-98feb90310ae" containerName="payload-pre-processing" containerID="cri-o://a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55" gracePeriod=30
Jul 26 18:41:02.769436 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.769411    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:41:02.784857 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.784837    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:41:02.813458 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.813429    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlvt5\" (UniqueName: \"kubernetes.io/projected/55fd0d33-c2c5-4200-b4dc-94e98323f22d-kube-api-access-xlvt5\") pod \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\" (UID: \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\") "
Jul 26 18:41:02.813646 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.813498    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55fd0d33-c2c5-4200-b4dc-94e98323f22d-plugins-config-volume\") pod \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\" (UID: \"55fd0d33-c2c5-4200-b4dc-94e98323f22d\") "
Jul 26 18:41:02.813869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.813839    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55fd0d33-c2c5-4200-b4dc-94e98323f22d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55fd0d33-c2c5-4200-b4dc-94e98323f22d" (UID: "55fd0d33-c2c5-4200-b4dc-94e98323f22d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:02.815526 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.815505    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55fd0d33-c2c5-4200-b4dc-94e98323f22d-kube-api-access-xlvt5" (OuterVolumeSpecName: "kube-api-access-xlvt5") pod "55fd0d33-c2c5-4200-b4dc-94e98323f22d" (UID: "55fd0d33-c2c5-4200-b4dc-94e98323f22d"). InnerVolumeSpecName "kube-api-access-xlvt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:02.915106 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.915053    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4zvd\" (UniqueName: \"kubernetes.io/projected/37645b37-2732-4974-944a-98feb90310ae-kube-api-access-b4zvd\") pod \"37645b37-2732-4974-944a-98feb90310ae\" (UID: \"37645b37-2732-4974-944a-98feb90310ae\") "
Jul 26 18:41:02.915287 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.915128    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37645b37-2732-4974-944a-98feb90310ae-plugins-config-volume\") pod \"37645b37-2732-4974-944a-98feb90310ae\" (UID: \"37645b37-2732-4974-944a-98feb90310ae\") "
Jul 26 18:41:02.915369 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.915350    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlvt5\" (UniqueName: \"kubernetes.io/projected/55fd0d33-c2c5-4200-b4dc-94e98323f22d-kube-api-access-xlvt5\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:41:02.915411 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.915376    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55fd0d33-c2c5-4200-b4dc-94e98323f22d-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:41:02.915572 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.915543    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37645b37-2732-4974-944a-98feb90310ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "37645b37-2732-4974-944a-98feb90310ae" (UID: "37645b37-2732-4974-944a-98feb90310ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:02.917117 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:02.917092    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37645b37-2732-4974-944a-98feb90310ae-kube-api-access-b4zvd" (OuterVolumeSpecName: "kube-api-access-b4zvd") pod "37645b37-2732-4974-944a-98feb90310ae" (UID: "37645b37-2732-4974-944a-98feb90310ae"). InnerVolumeSpecName "kube-api-access-b4zvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:03.016671 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.016553    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4zvd\" (UniqueName: \"kubernetes.io/projected/37645b37-2732-4974-944a-98feb90310ae-kube-api-access-b4zvd\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:41:03.016671 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.016586    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37645b37-2732-4974-944a-98feb90310ae-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:41:03.463412 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.463378    2559 generic.go:356] "Generic (PLEG): container finished" podID="55fd0d33-c2c5-4200-b4dc-94e98323f22d" containerID="78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6" exitCode=0
Jul 26 18:41:03.463601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.463422    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"
Jul 26 18:41:03.463601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.463471    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9" event={"ID":"55fd0d33-c2c5-4200-b4dc-94e98323f22d","Type":"ContainerDied","Data":"78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6"}
Jul 26 18:41:03.463601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.463524    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9" event={"ID":"55fd0d33-c2c5-4200-b4dc-94e98323f22d","Type":"ContainerDied","Data":"3c8164370e4e3b17c255743f6753b9112661b44251f0cb3e66c72e23a775591f"}
Jul 26 18:41:03.463601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.463548    2559 scope.go:117] "RemoveContainer" containerID="78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6"
Jul 26 18:41:03.465013 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.464990    2559 generic.go:356] "Generic (PLEG): container finished" podID="37645b37-2732-4974-944a-98feb90310ae" containerID="a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55" exitCode=0
Jul 26 18:41:03.465113 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.465053    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"
Jul 26 18:41:03.465113 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.465062    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v" event={"ID":"37645b37-2732-4974-944a-98feb90310ae","Type":"ContainerDied","Data":"a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55"}
Jul 26 18:41:03.465208 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.465129    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v" event={"ID":"37645b37-2732-4974-944a-98feb90310ae","Type":"ContainerDied","Data":"5b99409445064d8523d6609ed7595c26622b6e7d44684c9e5642a2c0ba9b2f17"}
Jul 26 18:41:03.477499 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.477473    2559 scope.go:117] "RemoveContainer" containerID="78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6"
Jul 26 18:41:03.477809 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:41:03.477790    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6\": container with ID starting with 78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6 not found: ID does not exist" containerID="78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6"
Jul 26 18:41:03.477865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.477817    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6"} err="failed to get container status \"78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6\": rpc error: code = NotFound desc = could not find container \"78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6\": container with ID starting with 78a7a4fdda3afeb76363f4e71a83d0c18bf7e4b455c4b7355d6b4e2f7032d4a6 not found: ID does not exist"
Jul 26 18:41:03.477865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.477834    2559 scope.go:117] "RemoveContainer" containerID="a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55"
Jul 26 18:41:03.483826 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.483802    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"]
Jul 26 18:41:03.487157 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.487142    2559 scope.go:117] "RemoveContainer" containerID="a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55"
Jul 26 18:41:03.487452 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:41:03.487433    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55\": container with ID starting with a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55 not found: ID does not exist" containerID="a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55"
Jul 26 18:41:03.487510 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.487459    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55"} err="failed to get container status \"a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55\": rpc error: code = NotFound desc = could not find container \"a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55\": container with ID starting with a6085e0110f2cf985e149012655167f0a5d6eefbb024416703ac5e8f09d05b55 not found: ID does not exist"
Jul 26 18:41:03.488528 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.488508    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-48526096-6778879b64-bfmb9"]
Jul 26 18:41:03.502785 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.502757    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"]
Jul 26 18:41:03.506196 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:03.506172    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-48526096-b94fd4948-m6x5v"]
Jul 26 18:41:05.176673 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:05.176639    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37645b37-2732-4974-944a-98feb90310ae" path="/var/lib/kubelet/pods/37645b37-2732-4974-944a-98feb90310ae/volumes"
Jul 26 18:41:05.177049 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:05.177003    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55fd0d33-c2c5-4200-b4dc-94e98323f22d" path="/var/lib/kubelet/pods/55fd0d33-c2c5-4200-b4dc-94e98323f22d/volumes"
Jul 26 18:41:12.388087 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.388055    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"]
Jul 26 18:41:12.388476 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.388427    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37645b37-2732-4974-944a-98feb90310ae" containerName="payload-pre-processing"
Jul 26 18:41:12.388476 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.388438    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="37645b37-2732-4974-944a-98feb90310ae" containerName="payload-pre-processing"
Jul 26 18:41:12.388476 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.388453    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55fd0d33-c2c5-4200-b4dc-94e98323f22d" containerName="payload-processing"
Jul 26 18:41:12.388476 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.388459    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="55fd0d33-c2c5-4200-b4dc-94e98323f22d" containerName="payload-processing"
Jul 26 18:41:12.388601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.388522    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="37645b37-2732-4974-944a-98feb90310ae" containerName="payload-pre-processing"
Jul 26 18:41:12.388601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.388530    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="55fd0d33-c2c5-4200-b4dc-94e98323f22d" containerName="payload-processing"
Jul 26 18:41:12.391387 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.391368    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:12.395902 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.395875    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-83ff5c8f\""
Jul 26 18:41:12.396022 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.395947    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-83ff5c8f-dockercfg-hcf64\""
Jul 26 18:41:12.412867 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.412843    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"]
Jul 26 18:41:12.414571 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.414551    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"]
Jul 26 18:41:12.418056 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.418038    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:12.429271 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.429246    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"]
Jul 26 18:41:12.507531 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.507501    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf40a555-46da-460a-a5c5-19ef1ff7aed8-plugins-config-volume\") pod \"payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz\" (UID: \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\") " pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:12.507729 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.507590    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn6sr\" (UniqueName: \"kubernetes.io/projected/bf40a555-46da-460a-a5c5-19ef1ff7aed8-kube-api-access-gn6sr\") pod \"payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz\" (UID: \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\") " pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:12.507802 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.507737    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbgv5\" (UniqueName: \"kubernetes.io/projected/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-kube-api-access-qbgv5\") pod \"payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw\" (UID: \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:12.507802 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.507767    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw\" (UID: \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:12.609205 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.609172    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn6sr\" (UniqueName: \"kubernetes.io/projected/bf40a555-46da-460a-a5c5-19ef1ff7aed8-kube-api-access-gn6sr\") pod \"payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz\" (UID: \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\") " pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:12.609359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.609247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbgv5\" (UniqueName: \"kubernetes.io/projected/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-kube-api-access-qbgv5\") pod \"payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw\" (UID: \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:12.609359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.609267    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw\" (UID: \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:12.609359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.609294    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf40a555-46da-460a-a5c5-19ef1ff7aed8-plugins-config-volume\") pod \"payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz\" (UID: \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\") " pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:12.609997 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.609980    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf40a555-46da-460a-a5c5-19ef1ff7aed8-plugins-config-volume\") pod \"payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz\" (UID: \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\") " pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:12.610041 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.610029    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw\" (UID: \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:12.617927 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.617901    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbgv5\" (UniqueName: \"kubernetes.io/projected/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-kube-api-access-qbgv5\") pod \"payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw\" (UID: \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:12.618039 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.617980    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn6sr\" (UniqueName: \"kubernetes.io/projected/bf40a555-46da-460a-a5c5-19ef1ff7aed8-kube-api-access-gn6sr\") pod \"payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz\" (UID: \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\") " pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:12.701598 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.701572    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:12.732131 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.732092    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:12.858300 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:12.858223    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"]
Jul 26 18:41:12.861253 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:41:12.861221    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d406df_d207_4d9c_82e2_e2d83c38b7ee.slice/crio-2bad40a2ea892af32ae034dd077d43d99708b633c79716b34b1be2b6a16d1d66 WatchSource:0}: Error finding container 2bad40a2ea892af32ae034dd077d43d99708b633c79716b34b1be2b6a16d1d66: Status 404 returned error can't find the container with id 2bad40a2ea892af32ae034dd077d43d99708b633c79716b34b1be2b6a16d1d66
Jul 26 18:41:13.089489 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:13.089466    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"]
Jul 26 18:41:13.091852 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:41:13.091822    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf40a555_46da_460a_a5c5_19ef1ff7aed8.slice/crio-574b1d6a149b09c4934607c33dae761363e401573914bb1d3b7d81c3ddbabd34 WatchSource:0}: Error finding container 574b1d6a149b09c4934607c33dae761363e401573914bb1d3b7d81c3ddbabd34: Status 404 returned error can't find the container with id 574b1d6a149b09c4934607c33dae761363e401573914bb1d3b7d81c3ddbabd34
Jul 26 18:41:13.515298 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:13.515266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw" event={"ID":"d0d406df-d207-4d9c-82e2-e2d83c38b7ee","Type":"ContainerStarted","Data":"e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d"}
Jul 26 18:41:13.515845 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:13.515813    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw" event={"ID":"d0d406df-d207-4d9c-82e2-e2d83c38b7ee","Type":"ContainerStarted","Data":"2bad40a2ea892af32ae034dd077d43d99708b633c79716b34b1be2b6a16d1d66"}
Jul 26 18:41:13.515915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:13.515890    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:13.523667 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:13.523640    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz" event={"ID":"bf40a555-46da-460a-a5c5-19ef1ff7aed8","Type":"ContainerStarted","Data":"fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b"}
Jul 26 18:41:13.523801 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:13.523713    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz" event={"ID":"bf40a555-46da-460a-a5c5-19ef1ff7aed8","Type":"ContainerStarted","Data":"574b1d6a149b09c4934607c33dae761363e401573914bb1d3b7d81c3ddbabd34"}
Jul 26 18:41:13.523801 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:13.523735    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:13.540443 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:13.540397    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw" podStartSLOduration=1.5403839110000002 podStartE2EDuration="1.540383911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:41:13.537716627 +0000 UTC m=+1744.862238725" watchObservedRunningTime="2026-07-26 18:41:13.540383911 +0000 UTC m=+1744.864906009"
Jul 26 18:41:13.564912 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:13.564873    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz" podStartSLOduration=1.564860994 podStartE2EDuration="1.564860994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:41:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:41:13.562705198 +0000 UTC m=+1744.887227295" watchObservedRunningTime="2026-07-26 18:41:13.564860994 +0000 UTC m=+1744.889383094"
Jul 26 18:41:24.531226 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:24.531181    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:24.531675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:24.531344    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:46.704472 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:46.704439    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"]
Jul 26 18:41:46.705099 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:46.704738    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz" podUID="bf40a555-46da-460a-a5c5-19ef1ff7aed8" containerName="payload-processing" containerID="cri-o://fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b" gracePeriod=30
Jul 26 18:41:46.711480 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:46.711454    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"]
Jul 26 18:41:46.711723 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:46.711676    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw" podUID="d0d406df-d207-4d9c-82e2-e2d83c38b7ee" containerName="payload-pre-processing" containerID="cri-o://e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d" gracePeriod=30
Jul 26 18:41:46.983117 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:46.983089    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:46.986053 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:46.986033    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:47.139220 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.139187    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-plugins-config-volume\") pod \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\" (UID: \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\") "
Jul 26 18:41:47.139405 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.139233    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbgv5\" (UniqueName: \"kubernetes.io/projected/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-kube-api-access-qbgv5\") pod \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\" (UID: \"d0d406df-d207-4d9c-82e2-e2d83c38b7ee\") "
Jul 26 18:41:47.139405 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.139255    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf40a555-46da-460a-a5c5-19ef1ff7aed8-plugins-config-volume\") pod \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\" (UID: \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\") "
Jul 26 18:41:47.139405 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.139296    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn6sr\" (UniqueName: \"kubernetes.io/projected/bf40a555-46da-460a-a5c5-19ef1ff7aed8-kube-api-access-gn6sr\") pod \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\" (UID: \"bf40a555-46da-460a-a5c5-19ef1ff7aed8\") "
Jul 26 18:41:47.139649 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.139603    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d0d406df-d207-4d9c-82e2-e2d83c38b7ee" (UID: "d0d406df-d207-4d9c-82e2-e2d83c38b7ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:47.139741 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.139719    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf40a555-46da-460a-a5c5-19ef1ff7aed8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bf40a555-46da-460a-a5c5-19ef1ff7aed8" (UID: "bf40a555-46da-460a-a5c5-19ef1ff7aed8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:41:47.141394 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.141368    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf40a555-46da-460a-a5c5-19ef1ff7aed8-kube-api-access-gn6sr" (OuterVolumeSpecName: "kube-api-access-gn6sr") pod "bf40a555-46da-460a-a5c5-19ef1ff7aed8" (UID: "bf40a555-46da-460a-a5c5-19ef1ff7aed8"). InnerVolumeSpecName "kube-api-access-gn6sr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:47.141692 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.141673    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-kube-api-access-qbgv5" (OuterVolumeSpecName: "kube-api-access-qbgv5") pod "d0d406df-d207-4d9c-82e2-e2d83c38b7ee" (UID: "d0d406df-d207-4d9c-82e2-e2d83c38b7ee"). InnerVolumeSpecName "kube-api-access-qbgv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:41:47.240435 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.240368    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn6sr\" (UniqueName: \"kubernetes.io/projected/bf40a555-46da-460a-a5c5-19ef1ff7aed8-kube-api-access-gn6sr\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:41:47.240435 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.240390    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:41:47.240435 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.240400    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbgv5\" (UniqueName: \"kubernetes.io/projected/d0d406df-d207-4d9c-82e2-e2d83c38b7ee-kube-api-access-qbgv5\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:41:47.240435 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.240409    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf40a555-46da-460a-a5c5-19ef1ff7aed8-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:41:47.689514 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.689476    2559 generic.go:356] "Generic (PLEG): container finished" podID="bf40a555-46da-460a-a5c5-19ef1ff7aed8" containerID="fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b" exitCode=0
Jul 26 18:41:47.689701 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.689525    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"
Jul 26 18:41:47.689701 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.689564    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz" event={"ID":"bf40a555-46da-460a-a5c5-19ef1ff7aed8","Type":"ContainerDied","Data":"fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b"}
Jul 26 18:41:47.689701 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.689638    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz" event={"ID":"bf40a555-46da-460a-a5c5-19ef1ff7aed8","Type":"ContainerDied","Data":"574b1d6a149b09c4934607c33dae761363e401573914bb1d3b7d81c3ddbabd34"}
Jul 26 18:41:47.689701 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.689666    2559 scope.go:117] "RemoveContainer" containerID="fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b"
Jul 26 18:41:47.691110 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.691082    2559 generic.go:356] "Generic (PLEG): container finished" podID="d0d406df-d207-4d9c-82e2-e2d83c38b7ee" containerID="e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d" exitCode=0
Jul 26 18:41:47.691110 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.691119    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw" event={"ID":"d0d406df-d207-4d9c-82e2-e2d83c38b7ee","Type":"ContainerDied","Data":"e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d"}
Jul 26 18:41:47.691302 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.691130    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"
Jul 26 18:41:47.691302 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.691156    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw" event={"ID":"d0d406df-d207-4d9c-82e2-e2d83c38b7ee","Type":"ContainerDied","Data":"2bad40a2ea892af32ae034dd077d43d99708b633c79716b34b1be2b6a16d1d66"}
Jul 26 18:41:47.698421 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.698405    2559 scope.go:117] "RemoveContainer" containerID="fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b"
Jul 26 18:41:47.698718 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:41:47.698687    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b\": container with ID starting with fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b not found: ID does not exist" containerID="fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b"
Jul 26 18:41:47.698817 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.698729    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b"} err="failed to get container status \"fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b\": rpc error: code = NotFound desc = could not find container \"fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b\": container with ID starting with fb882ee9dc9b7aa2d0472a32dc6669c3a56170ae0a95a59487f4b834bc6aa62b not found: ID does not exist"
Jul 26 18:41:47.698817 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.698750    2559 scope.go:117] "RemoveContainer" containerID="e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d"
Jul 26 18:41:47.706106 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.705951    2559 scope.go:117] "RemoveContainer" containerID="e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d"
Jul 26 18:41:47.706337 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:41:47.706206    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d\": container with ID starting with e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d not found: ID does not exist" containerID="e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d"
Jul 26 18:41:47.706337 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.706229    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d"} err="failed to get container status \"e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d\": rpc error: code = NotFound desc = could not find container \"e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d\": container with ID starting with e31d10e2e29ee3e5224307978dad3c874fd238665a1cca4fd200ee018b8ac79d not found: ID does not exist"
Jul 26 18:41:47.709886 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.709863    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"]
Jul 26 18:41:47.717103 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.717082    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-83ff5c8f-6484fb9f78-fj9cz"]
Jul 26 18:41:47.727482 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.727464    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"]
Jul 26 18:41:47.734499 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:47.734480    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-83ff5c8f-6cc586b676-l89jw"]
Jul 26 18:41:49.178459 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:49.178425    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf40a555-46da-460a-a5c5-19ef1ff7aed8" path="/var/lib/kubelet/pods/bf40a555-46da-460a-a5c5-19ef1ff7aed8/volumes"
Jul 26 18:41:49.179002 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:41:49.178982    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0d406df-d207-4d9c-82e2-e2d83c38b7ee" path="/var/lib/kubelet/pods/d0d406df-d207-4d9c-82e2-e2d83c38b7ee/volumes"
Jul 26 18:42:09.890073 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:42:09.889965    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:42:09.897462 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:42:09.897439    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:43:33.433806 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.433718    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"]
Jul 26 18:43:33.434332 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.434315    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf40a555-46da-460a-a5c5-19ef1ff7aed8" containerName="payload-processing"
Jul 26 18:43:33.434380 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.434336    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf40a555-46da-460a-a5c5-19ef1ff7aed8" containerName="payload-processing"
Jul 26 18:43:33.434380 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.434358    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0d406df-d207-4d9c-82e2-e2d83c38b7ee" containerName="payload-pre-processing"
Jul 26 18:43:33.434380 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.434368    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0d406df-d207-4d9c-82e2-e2d83c38b7ee" containerName="payload-pre-processing"
Jul 26 18:43:33.434469 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.434462    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf40a555-46da-460a-a5c5-19ef1ff7aed8" containerName="payload-processing"
Jul 26 18:43:33.434504 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.434475    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0d406df-d207-4d9c-82e2-e2d83c38b7ee" containerName="payload-pre-processing"
Jul 26 18:43:33.437964 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.437941    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:33.440958 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.440935    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0dc66676\""
Jul 26 18:43:33.441062 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.440982    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0dc66676-dockercfg-fzszh\""
Jul 26 18:43:33.445694 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.445669    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"]
Jul 26 18:43:33.449697 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.449670    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"]
Jul 26 18:43:33.449825 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.449676    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:33.460989 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.460967    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"]
Jul 26 18:43:33.588969 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.588937    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df365f11-7e54-49b2-bdd6-d58ea057c989-plugins-config-volume\") pod \"payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt\" (UID: \"df365f11-7e54-49b2-bdd6-d58ea057c989\") " pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:33.589123 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.589043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904f462d-2a37-4919-bf5a-12b958ca7092-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh\" (UID: \"904f462d-2a37-4919-bf5a-12b958ca7092\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:33.589123 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.589063    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v26j\" (UniqueName: \"kubernetes.io/projected/df365f11-7e54-49b2-bdd6-d58ea057c989-kube-api-access-8v26j\") pod \"payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt\" (UID: \"df365f11-7e54-49b2-bdd6-d58ea057c989\") " pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:33.589123 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.589086    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksdv9\" (UniqueName: \"kubernetes.io/projected/904f462d-2a37-4919-bf5a-12b958ca7092-kube-api-access-ksdv9\") pod \"payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh\" (UID: \"904f462d-2a37-4919-bf5a-12b958ca7092\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:33.690165 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.690084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904f462d-2a37-4919-bf5a-12b958ca7092-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh\" (UID: \"904f462d-2a37-4919-bf5a-12b958ca7092\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:33.690165 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.690126    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v26j\" (UniqueName: \"kubernetes.io/projected/df365f11-7e54-49b2-bdd6-d58ea057c989-kube-api-access-8v26j\") pod \"payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt\" (UID: \"df365f11-7e54-49b2-bdd6-d58ea057c989\") " pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:33.690165 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.690155    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksdv9\" (UniqueName: \"kubernetes.io/projected/904f462d-2a37-4919-bf5a-12b958ca7092-kube-api-access-ksdv9\") pod \"payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh\" (UID: \"904f462d-2a37-4919-bf5a-12b958ca7092\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:33.690369 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.690220    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df365f11-7e54-49b2-bdd6-d58ea057c989-plugins-config-volume\") pod \"payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt\" (UID: \"df365f11-7e54-49b2-bdd6-d58ea057c989\") " pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:33.691129 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.691108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df365f11-7e54-49b2-bdd6-d58ea057c989-plugins-config-volume\") pod \"payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt\" (UID: \"df365f11-7e54-49b2-bdd6-d58ea057c989\") " pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:33.691177 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.691108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904f462d-2a37-4919-bf5a-12b958ca7092-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh\" (UID: \"904f462d-2a37-4919-bf5a-12b958ca7092\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:33.702772 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.702746    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v26j\" (UniqueName: \"kubernetes.io/projected/df365f11-7e54-49b2-bdd6-d58ea057c989-kube-api-access-8v26j\") pod \"payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt\" (UID: \"df365f11-7e54-49b2-bdd6-d58ea057c989\") " pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:33.704965 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.704939    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksdv9\" (UniqueName: \"kubernetes.io/projected/904f462d-2a37-4919-bf5a-12b958ca7092-kube-api-access-ksdv9\") pod \"payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh\" (UID: \"904f462d-2a37-4919-bf5a-12b958ca7092\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:33.751100 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.751072    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:33.760085 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.760061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:33.903419 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.903392    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"]
Jul 26 18:43:33.905071 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:43:33.905038    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod904f462d_2a37_4919_bf5a_12b958ca7092.slice/crio-19dd6453d928c21718cbeb3c26c8419a7c336c3957640fea84d2e5dc4a577b31 WatchSource:0}: Error finding container 19dd6453d928c21718cbeb3c26c8419a7c336c3957640fea84d2e5dc4a577b31: Status 404 returned error can't find the container with id 19dd6453d928c21718cbeb3c26c8419a7c336c3957640fea84d2e5dc4a577b31
Jul 26 18:43:33.927015 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:33.926961    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"]
Jul 26 18:43:33.929589 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:43:33.929563    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf365f11_7e54_49b2_bdd6_d58ea057c989.slice/crio-2afce84d1c4e53ee04ab53873ce77dd5e33b2b3e979e7791655c3384d3c0a73b WatchSource:0}: Error finding container 2afce84d1c4e53ee04ab53873ce77dd5e33b2b3e979e7791655c3384d3c0a73b: Status 404 returned error can't find the container with id 2afce84d1c4e53ee04ab53873ce77dd5e33b2b3e979e7791655c3384d3c0a73b
Jul 26 18:43:34.196201 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:34.196162    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt" event={"ID":"df365f11-7e54-49b2-bdd6-d58ea057c989","Type":"ContainerStarted","Data":"51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b"}
Jul 26 18:43:34.196381 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:34.196218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt" event={"ID":"df365f11-7e54-49b2-bdd6-d58ea057c989","Type":"ContainerStarted","Data":"2afce84d1c4e53ee04ab53873ce77dd5e33b2b3e979e7791655c3384d3c0a73b"}
Jul 26 18:43:34.196381 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:34.196275    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:34.197839 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:34.197817    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh" event={"ID":"904f462d-2a37-4919-bf5a-12b958ca7092","Type":"ContainerStarted","Data":"dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a"}
Jul 26 18:43:34.197923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:34.197855    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh" event={"ID":"904f462d-2a37-4919-bf5a-12b958ca7092","Type":"ContainerStarted","Data":"19dd6453d928c21718cbeb3c26c8419a7c336c3957640fea84d2e5dc4a577b31"}
Jul 26 18:43:34.197923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:34.197883    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:34.214747 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:34.214674    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt" podStartSLOduration=1.214661122 podStartE2EDuration="1.214661122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:43:34.212467809 +0000 UTC m=+1885.536989941" watchObservedRunningTime="2026-07-26 18:43:34.214661122 +0000 UTC m=+1885.539183246"
Jul 26 18:43:34.229273 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:34.229234    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh" podStartSLOduration=1.229222607 podStartE2EDuration="1.229222607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:43:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:43:34.227304111 +0000 UTC m=+1885.551826209" watchObservedRunningTime="2026-07-26 18:43:34.229222607 +0000 UTC m=+1885.553744703"
Jul 26 18:43:45.204415 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:45.204380    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:45.205178 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:45.204910    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:54.651721 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.651682    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"]
Jul 26 18:43:54.652167 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.651958    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt" podUID="df365f11-7e54-49b2-bdd6-d58ea057c989" containerName="payload-processing" containerID="cri-o://51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b" gracePeriod=30
Jul 26 18:43:54.663169 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.663137    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"]
Jul 26 18:43:54.663487 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.663396    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh" podUID="904f462d-2a37-4919-bf5a-12b958ca7092" containerName="payload-pre-processing" containerID="cri-o://dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a" gracePeriod=30
Jul 26 18:43:54.939006 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.938985    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:54.943813 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.943785    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:54.983506 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.983481    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v26j\" (UniqueName: \"kubernetes.io/projected/df365f11-7e54-49b2-bdd6-d58ea057c989-kube-api-access-8v26j\") pod \"df365f11-7e54-49b2-bdd6-d58ea057c989\" (UID: \"df365f11-7e54-49b2-bdd6-d58ea057c989\") "
Jul 26 18:43:54.983676 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.983531    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df365f11-7e54-49b2-bdd6-d58ea057c989-plugins-config-volume\") pod \"df365f11-7e54-49b2-bdd6-d58ea057c989\" (UID: \"df365f11-7e54-49b2-bdd6-d58ea057c989\") "
Jul 26 18:43:54.983917 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.983894    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df365f11-7e54-49b2-bdd6-d58ea057c989-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df365f11-7e54-49b2-bdd6-d58ea057c989" (UID: "df365f11-7e54-49b2-bdd6-d58ea057c989"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:43:54.985710 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:54.985676    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df365f11-7e54-49b2-bdd6-d58ea057c989-kube-api-access-8v26j" (OuterVolumeSpecName: "kube-api-access-8v26j") pod "df365f11-7e54-49b2-bdd6-d58ea057c989" (UID: "df365f11-7e54-49b2-bdd6-d58ea057c989"). InnerVolumeSpecName "kube-api-access-8v26j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:43:55.084797 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.084773    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksdv9\" (UniqueName: \"kubernetes.io/projected/904f462d-2a37-4919-bf5a-12b958ca7092-kube-api-access-ksdv9\") pod \"904f462d-2a37-4919-bf5a-12b958ca7092\" (UID: \"904f462d-2a37-4919-bf5a-12b958ca7092\") "
Jul 26 18:43:55.084941 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.084820    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904f462d-2a37-4919-bf5a-12b958ca7092-plugins-config-volume\") pod \"904f462d-2a37-4919-bf5a-12b958ca7092\" (UID: \"904f462d-2a37-4919-bf5a-12b958ca7092\") "
Jul 26 18:43:55.085045 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.085029    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v26j\" (UniqueName: \"kubernetes.io/projected/df365f11-7e54-49b2-bdd6-d58ea057c989-kube-api-access-8v26j\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.085082 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.085053    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df365f11-7e54-49b2-bdd6-d58ea057c989-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.085226 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.085209    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/904f462d-2a37-4919-bf5a-12b958ca7092-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "904f462d-2a37-4919-bf5a-12b958ca7092" (UID: "904f462d-2a37-4919-bf5a-12b958ca7092"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:43:55.086724 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.086701    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/904f462d-2a37-4919-bf5a-12b958ca7092-kube-api-access-ksdv9" (OuterVolumeSpecName: "kube-api-access-ksdv9") pod "904f462d-2a37-4919-bf5a-12b958ca7092" (UID: "904f462d-2a37-4919-bf5a-12b958ca7092"). InnerVolumeSpecName "kube-api-access-ksdv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:43:55.186280 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.186064    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksdv9\" (UniqueName: \"kubernetes.io/projected/904f462d-2a37-4919-bf5a-12b958ca7092-kube-api-access-ksdv9\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.186280 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.186091    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/904f462d-2a37-4919-bf5a-12b958ca7092-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:43:55.301131 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.301091    2559 generic.go:356] "Generic (PLEG): container finished" podID="df365f11-7e54-49b2-bdd6-d58ea057c989" containerID="51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b" exitCode=0
Jul 26 18:43:55.301271 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.301133    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"
Jul 26 18:43:55.301271 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.301160    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt" event={"ID":"df365f11-7e54-49b2-bdd6-d58ea057c989","Type":"ContainerDied","Data":"51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b"}
Jul 26 18:43:55.301271 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.301218    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt" event={"ID":"df365f11-7e54-49b2-bdd6-d58ea057c989","Type":"ContainerDied","Data":"2afce84d1c4e53ee04ab53873ce77dd5e33b2b3e979e7791655c3384d3c0a73b"}
Jul 26 18:43:55.301271 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.301245    2559 scope.go:117] "RemoveContainer" containerID="51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b"
Jul 26 18:43:55.302492 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.302468    2559 generic.go:356] "Generic (PLEG): container finished" podID="904f462d-2a37-4919-bf5a-12b958ca7092" containerID="dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a" exitCode=0
Jul 26 18:43:55.302610 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.302495    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh" event={"ID":"904f462d-2a37-4919-bf5a-12b958ca7092","Type":"ContainerDied","Data":"dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a"}
Jul 26 18:43:55.302610 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.302531    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"
Jul 26 18:43:55.302610 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.302548    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh" event={"ID":"904f462d-2a37-4919-bf5a-12b958ca7092","Type":"ContainerDied","Data":"19dd6453d928c21718cbeb3c26c8419a7c336c3957640fea84d2e5dc4a577b31"}
Jul 26 18:43:55.310194 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.310168    2559 scope.go:117] "RemoveContainer" containerID="51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b"
Jul 26 18:43:55.310434 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:43:55.310415    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b\": container with ID starting with 51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b not found: ID does not exist" containerID="51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b"
Jul 26 18:43:55.310496 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.310445    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b"} err="failed to get container status \"51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b\": rpc error: code = NotFound desc = could not find container \"51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b\": container with ID starting with 51076b46ad128d2823d4a8a0d6e00a700d6f82df401fef43701fe59daf8ae06b not found: ID does not exist"
Jul 26 18:43:55.310496 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.310469    2559 scope.go:117] "RemoveContainer" containerID="dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a"
Jul 26 18:43:55.318598 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.318580    2559 scope.go:117] "RemoveContainer" containerID="dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a"
Jul 26 18:43:55.318906 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:43:55.318886    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a\": container with ID starting with dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a not found: ID does not exist" containerID="dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a"
Jul 26 18:43:55.318991 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.318911    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a"} err="failed to get container status \"dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a\": rpc error: code = NotFound desc = could not find container \"dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a\": container with ID starting with dff1108933381fcd6fc6379c2485d8ec37a9a35873be5673ca33504c01e1fd0a not found: ID does not exist"
Jul 26 18:43:55.320051 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.320033    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"]
Jul 26 18:43:55.324004 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.323985    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0dc66676-75fcdf57d-fxsjt"]
Jul 26 18:43:55.333805 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.333782    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"]
Jul 26 18:43:55.339204 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:55.339182    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0dc66676-558fc9c888-g7krh"]
Jul 26 18:43:57.176689 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:57.176656    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="904f462d-2a37-4919-bf5a-12b958ca7092" path="/var/lib/kubelet/pods/904f462d-2a37-4919-bf5a-12b958ca7092/volumes"
Jul 26 18:43:57.177063 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:43:57.177015    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df365f11-7e54-49b2-bdd6-d58ea057c989" path="/var/lib/kubelet/pods/df365f11-7e54-49b2-bdd6-d58ea057c989/volumes"
Jul 26 18:44:28.430196 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.430156    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"]
Jul 26 18:44:28.430670 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.430529    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="904f462d-2a37-4919-bf5a-12b958ca7092" containerName="payload-pre-processing"
Jul 26 18:44:28.430670 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.430540    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="904f462d-2a37-4919-bf5a-12b958ca7092" containerName="payload-pre-processing"
Jul 26 18:44:28.430670 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.430560    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df365f11-7e54-49b2-bdd6-d58ea057c989" containerName="payload-processing"
Jul 26 18:44:28.430670 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.430565    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="df365f11-7e54-49b2-bdd6-d58ea057c989" containerName="payload-processing"
Jul 26 18:44:28.430670 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.430634    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="904f462d-2a37-4919-bf5a-12b958ca7092" containerName="payload-pre-processing"
Jul 26 18:44:28.430670 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.430646    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="df365f11-7e54-49b2-bdd6-d58ea057c989" containerName="payload-processing"
Jul 26 18:44:28.433843 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.433823    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:28.436984 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.436964    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-1d6ece93\""
Jul 26 18:44:28.445259 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.445238    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"]
Jul 26 18:44:28.447117 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.447091    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"]
Jul 26 18:44:28.450829 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.450810    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:28.453877 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.453858    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-1d6ece93-dockercfg-xz5gx\""
Jul 26 18:44:28.461882 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.461856    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"]
Jul 26 18:44:28.487429 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.487404    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rr86\" (UniqueName: \"kubernetes.io/projected/261bed6d-b004-4304-bdc7-88438d79bae5-kube-api-access-7rr86\") pod \"payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj\" (UID: \"261bed6d-b004-4304-bdc7-88438d79bae5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:28.487542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.487441    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e064a15-60a6-489d-8721-912b7e7264e9-plugins-config-volume\") pod \"payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f\" (UID: \"5e064a15-60a6-489d-8721-912b7e7264e9\") " pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:28.487542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.487465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrxcm\" (UniqueName: \"kubernetes.io/projected/5e064a15-60a6-489d-8721-912b7e7264e9-kube-api-access-mrxcm\") pod \"payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f\" (UID: \"5e064a15-60a6-489d-8721-912b7e7264e9\") " pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:28.487542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.487531    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261bed6d-b004-4304-bdc7-88438d79bae5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj\" (UID: \"261bed6d-b004-4304-bdc7-88438d79bae5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:28.588493 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.588453    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rr86\" (UniqueName: \"kubernetes.io/projected/261bed6d-b004-4304-bdc7-88438d79bae5-kube-api-access-7rr86\") pod \"payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj\" (UID: \"261bed6d-b004-4304-bdc7-88438d79bae5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:28.588652 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.588498    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e064a15-60a6-489d-8721-912b7e7264e9-plugins-config-volume\") pod \"payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f\" (UID: \"5e064a15-60a6-489d-8721-912b7e7264e9\") " pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:28.588652 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.588540    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrxcm\" (UniqueName: \"kubernetes.io/projected/5e064a15-60a6-489d-8721-912b7e7264e9-kube-api-access-mrxcm\") pod \"payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f\" (UID: \"5e064a15-60a6-489d-8721-912b7e7264e9\") " pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:28.588652 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.588568    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261bed6d-b004-4304-bdc7-88438d79bae5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj\" (UID: \"261bed6d-b004-4304-bdc7-88438d79bae5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:28.589318 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.589298    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261bed6d-b004-4304-bdc7-88438d79bae5-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj\" (UID: \"261bed6d-b004-4304-bdc7-88438d79bae5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:28.589361 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.589300    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e064a15-60a6-489d-8721-912b7e7264e9-plugins-config-volume\") pod \"payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f\" (UID: \"5e064a15-60a6-489d-8721-912b7e7264e9\") " pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:28.597244 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.597217    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rr86\" (UniqueName: \"kubernetes.io/projected/261bed6d-b004-4304-bdc7-88438d79bae5-kube-api-access-7rr86\") pod \"payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj\" (UID: \"261bed6d-b004-4304-bdc7-88438d79bae5\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:28.597352 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.597220    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrxcm\" (UniqueName: \"kubernetes.io/projected/5e064a15-60a6-489d-8721-912b7e7264e9-kube-api-access-mrxcm\") pod \"payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f\" (UID: \"5e064a15-60a6-489d-8721-912b7e7264e9\") " pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:28.745260 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.745180    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:28.763519 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.763493    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:28.892692 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:28.892672    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"]
Jul 26 18:44:28.895806 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:44:28.895778    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261bed6d_b004_4304_bdc7_88438d79bae5.slice/crio-3e6519f376bb507e7cd32462bc1839d71aa22505a6f4e7b517eb7702945aaa1c WatchSource:0}: Error finding container 3e6519f376bb507e7cd32462bc1839d71aa22505a6f4e7b517eb7702945aaa1c: Status 404 returned error can't find the container with id 3e6519f376bb507e7cd32462bc1839d71aa22505a6f4e7b517eb7702945aaa1c
Jul 26 18:44:29.119173 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:29.119149    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"]
Jul 26 18:44:29.121512 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:44:29.121483    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e064a15_60a6_489d_8721_912b7e7264e9.slice/crio-778e324725dfebc30522c4eda329997d228b7d490213d0fb01d7f8e2968803d2 WatchSource:0}: Error finding container 778e324725dfebc30522c4eda329997d228b7d490213d0fb01d7f8e2968803d2: Status 404 returned error can't find the container with id 778e324725dfebc30522c4eda329997d228b7d490213d0fb01d7f8e2968803d2
Jul 26 18:44:29.474269 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:29.474229    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f" event={"ID":"5e064a15-60a6-489d-8721-912b7e7264e9","Type":"ContainerStarted","Data":"9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e"}
Jul 26 18:44:29.474757 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:29.474290    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f" event={"ID":"5e064a15-60a6-489d-8721-912b7e7264e9","Type":"ContainerStarted","Data":"778e324725dfebc30522c4eda329997d228b7d490213d0fb01d7f8e2968803d2"}
Jul 26 18:44:29.474757 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:29.474322    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:29.475860 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:29.475838    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj" event={"ID":"261bed6d-b004-4304-bdc7-88438d79bae5","Type":"ContainerStarted","Data":"ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b"}
Jul 26 18:44:29.475976 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:29.475880    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj" event={"ID":"261bed6d-b004-4304-bdc7-88438d79bae5","Type":"ContainerStarted","Data":"3e6519f376bb507e7cd32462bc1839d71aa22505a6f4e7b517eb7702945aaa1c"}
Jul 26 18:44:29.475976 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:29.475941    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:29.491605 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:29.491543    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f" podStartSLOduration=1.4915279909999999 podStartE2EDuration="1.491527991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:44:29.491225179 +0000 UTC m=+1940.815747279" watchObservedRunningTime="2026-07-26 18:44:29.491527991 +0000 UTC m=+1940.816050092"
Jul 26 18:44:29.507471 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:29.507429    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj" podStartSLOduration=1.507417298 podStartE2EDuration="1.507417298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:44:29.506424405 +0000 UTC m=+1940.830946504" watchObservedRunningTime="2026-07-26 18:44:29.507417298 +0000 UTC m=+1940.831939395"
Jul 26 18:44:40.483248 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:40.483203    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj" podUID="261bed6d-b004-4304-bdc7-88438d79bae5" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.46:9004: connect: connection refused"
Jul 26 18:44:40.483753 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:40.483542    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:44:49.494877 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.494846    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"]
Jul 26 18:44:49.498409 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.498391    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.501245 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.501226    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1d6ece93\"/\"default-dockercfg-6xg62\""
Jul 26 18:44:49.502555 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.502535    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1d6ece93\"/\"kube-root-ca.crt\""
Jul 26 18:44:49.502709 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.502627    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1d6ece93\"/\"e2e-lifecycle-model-1d6ece93-kserve-self-signed-certs\""
Jul 26 18:44:49.502709 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.502645    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1d6ece93\"/\"openshift-service-ca.crt\""
Jul 26 18:44:49.509459 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.509407    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"]
Jul 26 18:44:49.580065 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.580034    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-home\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.580232 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.580135    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-dshm\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.580232 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.580167    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tmp-dir\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.580232 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.580203    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqcrv\" (UniqueName: \"kubernetes.io/projected/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-kube-api-access-tqcrv\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.580451 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.580247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-model-cache\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.580451 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.580308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tls-certs\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.681135 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.681107    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-dshm\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.681135 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.681138    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tmp-dir\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.681376 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.681163    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqcrv\" (UniqueName: \"kubernetes.io/projected/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-kube-api-access-tqcrv\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.681376 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.681278    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-model-cache\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.681376 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.681339    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tls-certs\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.681376 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.681368    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-home\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.681581 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.681515    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tmp-dir\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.681657 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.681591    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-model-cache\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.681720 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.681701    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-home\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.683527 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.683502    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-dshm\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.684072 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.684055    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tls-certs\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.689562 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.689541    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqcrv\" (UniqueName: \"kubernetes.io/projected/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-kube-api-access-tqcrv\") pod \"e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") " pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.810678 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.810596    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:49.941865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:49.941841    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"]
Jul 26 18:44:49.943899 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:44:49.943869    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d389f5a_9db4_404c_8d4a_7f3f4c73108b.slice/crio-c2fbc4ef4d845eae77fff7321994fb3a21806e51ad430365426bbf2ab02b88c9 WatchSource:0}: Error finding container c2fbc4ef4d845eae77fff7321994fb3a21806e51ad430365426bbf2ab02b88c9: Status 404 returned error can't find the container with id c2fbc4ef4d845eae77fff7321994fb3a21806e51ad430365426bbf2ab02b88c9
Jul 26 18:44:50.482418 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:50.482386    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:44:50.577168 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:50.577136    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q" event={"ID":"4d389f5a-9db4-404c-8d4a-7f3f4c73108b","Type":"ContainerStarted","Data":"79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec"}
Jul 26 18:44:50.577538 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:50.577177    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q" event={"ID":"4d389f5a-9db4-404c-8d4a-7f3f4c73108b","Type":"ContainerStarted","Data":"c2fbc4ef4d845eae77fff7321994fb3a21806e51ad430365426bbf2ab02b88c9"}
Jul 26 18:44:50.598219 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:50.598161    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q" podStartSLOduration=1.598141188 podStartE2EDuration="1.598141188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:44:50.595878084 +0000 UTC m=+1961.920400180" watchObservedRunningTime="2026-07-26 18:44:50.598141188 +0000 UTC m=+1961.922663284"
Jul 26 18:44:59.811726 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:59.811689    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:59.812130 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:59.811766    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:44:59.824325 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:44:59.824304    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:45:00.640804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:00.640727    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:45:27.793098 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:27.793062    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"]
Jul 26 18:45:27.793754 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:27.793294    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f" podUID="5e064a15-60a6-489d-8721-912b7e7264e9" containerName="payload-processing" containerID="cri-o://9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e" gracePeriod=30
Jul 26 18:45:27.802755 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:27.802727    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"]
Jul 26 18:45:27.802976 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:27.802951    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj" podUID="261bed6d-b004-4304-bdc7-88438d79bae5" containerName="payload-pre-processing" containerID="cri-o://ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b" gracePeriod=30
Jul 26 18:45:28.046023 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.045957    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:45:28.071218 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.071193    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:45:28.141907 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.141874    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrxcm\" (UniqueName: \"kubernetes.io/projected/5e064a15-60a6-489d-8721-912b7e7264e9-kube-api-access-mrxcm\") pod \"5e064a15-60a6-489d-8721-912b7e7264e9\" (UID: \"5e064a15-60a6-489d-8721-912b7e7264e9\") "
Jul 26 18:45:28.142080 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.141915    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7rr86\" (UniqueName: \"kubernetes.io/projected/261bed6d-b004-4304-bdc7-88438d79bae5-kube-api-access-7rr86\") pod \"261bed6d-b004-4304-bdc7-88438d79bae5\" (UID: \"261bed6d-b004-4304-bdc7-88438d79bae5\") "
Jul 26 18:45:28.142080 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.141947    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261bed6d-b004-4304-bdc7-88438d79bae5-plugins-config-volume\") pod \"261bed6d-b004-4304-bdc7-88438d79bae5\" (UID: \"261bed6d-b004-4304-bdc7-88438d79bae5\") "
Jul 26 18:45:28.142080 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.141965    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e064a15-60a6-489d-8721-912b7e7264e9-plugins-config-volume\") pod \"5e064a15-60a6-489d-8721-912b7e7264e9\" (UID: \"5e064a15-60a6-489d-8721-912b7e7264e9\") "
Jul 26 18:45:28.142369 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.142344    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/261bed6d-b004-4304-bdc7-88438d79bae5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "261bed6d-b004-4304-bdc7-88438d79bae5" (UID: "261bed6d-b004-4304-bdc7-88438d79bae5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:45:28.142434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.142402    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e064a15-60a6-489d-8721-912b7e7264e9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e064a15-60a6-489d-8721-912b7e7264e9" (UID: "5e064a15-60a6-489d-8721-912b7e7264e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:45:28.144133 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.144109    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261bed6d-b004-4304-bdc7-88438d79bae5-kube-api-access-7rr86" (OuterVolumeSpecName: "kube-api-access-7rr86") pod "261bed6d-b004-4304-bdc7-88438d79bae5" (UID: "261bed6d-b004-4304-bdc7-88438d79bae5"). InnerVolumeSpecName "kube-api-access-7rr86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:45:28.144259 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.144230    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e064a15-60a6-489d-8721-912b7e7264e9-kube-api-access-mrxcm" (OuterVolumeSpecName: "kube-api-access-mrxcm") pod "5e064a15-60a6-489d-8721-912b7e7264e9" (UID: "5e064a15-60a6-489d-8721-912b7e7264e9"). InnerVolumeSpecName "kube-api-access-mrxcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:45:28.243821 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.243782    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrxcm\" (UniqueName: \"kubernetes.io/projected/5e064a15-60a6-489d-8721-912b7e7264e9-kube-api-access-mrxcm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:28.243821 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.243814    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rr86\" (UniqueName: \"kubernetes.io/projected/261bed6d-b004-4304-bdc7-88438d79bae5-kube-api-access-7rr86\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:28.243988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.243831    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/261bed6d-b004-4304-bdc7-88438d79bae5-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:28.243988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.243851    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e064a15-60a6-489d-8721-912b7e7264e9-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:28.766202 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.766160    2559 generic.go:356] "Generic (PLEG): container finished" podID="261bed6d-b004-4304-bdc7-88438d79bae5" containerID="ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b" exitCode=0
Jul 26 18:45:28.766397 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.766221    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"
Jul 26 18:45:28.766397 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.766253    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj" event={"ID":"261bed6d-b004-4304-bdc7-88438d79bae5","Type":"ContainerDied","Data":"ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b"}
Jul 26 18:45:28.766397 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.766319    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj" event={"ID":"261bed6d-b004-4304-bdc7-88438d79bae5","Type":"ContainerDied","Data":"3e6519f376bb507e7cd32462bc1839d71aa22505a6f4e7b517eb7702945aaa1c"}
Jul 26 18:45:28.766397 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.766347    2559 scope.go:117] "RemoveContainer" containerID="ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b"
Jul 26 18:45:28.767716 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.767688    2559 generic.go:356] "Generic (PLEG): container finished" podID="5e064a15-60a6-489d-8721-912b7e7264e9" containerID="9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e" exitCode=0
Jul 26 18:45:28.767883 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.767764    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f" event={"ID":"5e064a15-60a6-489d-8721-912b7e7264e9","Type":"ContainerDied","Data":"9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e"}
Jul 26 18:45:28.767883 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.767815    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f" event={"ID":"5e064a15-60a6-489d-8721-912b7e7264e9","Type":"ContainerDied","Data":"778e324725dfebc30522c4eda329997d228b7d490213d0fb01d7f8e2968803d2"}
Jul 26 18:45:28.767883 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.767881    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"
Jul 26 18:45:28.785335 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.781026    2559 scope.go:117] "RemoveContainer" containerID="ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b"
Jul 26 18:45:28.785335 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:45:28.784381    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b\": container with ID starting with ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b not found: ID does not exist" containerID="ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b"
Jul 26 18:45:28.785335 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.784425    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b"} err="failed to get container status \"ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b\": rpc error: code = NotFound desc = could not find container \"ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b\": container with ID starting with ebc75c9d3970104d3e27aa91b1cffefb5a5d90d9b73581468af482d84b08fc3b not found: ID does not exist"
Jul 26 18:45:28.785335 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.784451    2559 scope.go:117] "RemoveContainer" containerID="9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e"
Jul 26 18:45:28.796042 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.795843    2559 scope.go:117] "RemoveContainer" containerID="9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e"
Jul 26 18:45:28.796327 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:45:28.796261    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e\": container with ID starting with 9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e not found: ID does not exist" containerID="9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e"
Jul 26 18:45:28.796327 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.796288    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e"} err="failed to get container status \"9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e\": rpc error: code = NotFound desc = could not find container \"9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e\": container with ID starting with 9a7be0e7867ca20f0d0db0774b59de1121f97b04934e4e56457dea6f8b78864e not found: ID does not exist"
Jul 26 18:45:28.799567 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.799545    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"]
Jul 26 18:45:28.806457 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.806437    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-1d6ece93-5bffb86b64-rwqdj"]
Jul 26 18:45:28.816159 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.816138    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"]
Jul 26 18:45:28.823663 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:28.823640    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-1d6ece93-5764f5cd74-c865f"]
Jul 26 18:45:29.176312 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:29.176276    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="261bed6d-b004-4304-bdc7-88438d79bae5" path="/var/lib/kubelet/pods/261bed6d-b004-4304-bdc7-88438d79bae5/volumes"
Jul 26 18:45:29.176668 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:29.176654    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e064a15-60a6-489d-8721-912b7e7264e9" path="/var/lib/kubelet/pods/5e064a15-60a6-489d-8721-912b7e7264e9/volumes"
Jul 26 18:45:40.363885 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:40.363850    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"]
Jul 26 18:45:55.384217 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.384160    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q" podUID="4d389f5a-9db4-404c-8d4a-7f3f4c73108b" containerName="main" containerID="cri-o://79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec" gracePeriod=45
Jul 26 18:45:55.629336 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.629312    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:45:55.696429 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.696399    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tmp-dir\") pod \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") "
Jul 26 18:45:55.696592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.696459    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tls-certs\") pod \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") "
Jul 26 18:45:55.696592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.696508    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqcrv\" (UniqueName: \"kubernetes.io/projected/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-kube-api-access-tqcrv\") pod \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") "
Jul 26 18:45:55.696592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.696536    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-dshm\") pod \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") "
Jul 26 18:45:55.696592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.696569    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-home\") pod \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") "
Jul 26 18:45:55.696823 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.696655    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-model-cache\") pod \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\" (UID: \"4d389f5a-9db4-404c-8d4a-7f3f4c73108b\") "
Jul 26 18:45:55.696823 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.696690    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4d389f5a-9db4-404c-8d4a-7f3f4c73108b" (UID: "4d389f5a-9db4-404c-8d4a-7f3f4c73108b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:45:55.696928 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.696865    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-home" (OuterVolumeSpecName: "home") pod "4d389f5a-9db4-404c-8d4a-7f3f4c73108b" (UID: "4d389f5a-9db4-404c-8d4a-7f3f4c73108b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:45:55.696976 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.696916    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-model-cache" (OuterVolumeSpecName: "model-cache") pod "4d389f5a-9db4-404c-8d4a-7f3f4c73108b" (UID: "4d389f5a-9db4-404c-8d4a-7f3f4c73108b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:45:55.697034 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.697016    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:55.697074 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.697038    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:55.698783 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.698760    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-kube-api-access-tqcrv" (OuterVolumeSpecName: "kube-api-access-tqcrv") pod "4d389f5a-9db4-404c-8d4a-7f3f4c73108b" (UID: "4d389f5a-9db4-404c-8d4a-7f3f4c73108b"). InnerVolumeSpecName "kube-api-access-tqcrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:45:55.699094 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.699072    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-dshm" (OuterVolumeSpecName: "dshm") pod "4d389f5a-9db4-404c-8d4a-7f3f4c73108b" (UID: "4d389f5a-9db4-404c-8d4a-7f3f4c73108b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:45:55.699094 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.699079    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4d389f5a-9db4-404c-8d4a-7f3f4c73108b" (UID: "4d389f5a-9db4-404c-8d4a-7f3f4c73108b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:45:55.798443 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.798411    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:55.798443 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.798438    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:55.798443 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.798447    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqcrv\" (UniqueName: \"kubernetes.io/projected/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-kube-api-access-tqcrv\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:55.798443 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.798456    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d389f5a-9db4-404c-8d4a-7f3f4c73108b-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:45:55.907573 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.907537    2559 generic.go:356] "Generic (PLEG): container finished" podID="4d389f5a-9db4-404c-8d4a-7f3f4c73108b" containerID="79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec" exitCode=0
Jul 26 18:45:55.907769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.907589    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"
Jul 26 18:45:55.907769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.907636    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q" event={"ID":"4d389f5a-9db4-404c-8d4a-7f3f4c73108b","Type":"ContainerDied","Data":"79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec"}
Jul 26 18:45:55.907769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.907681    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q" event={"ID":"4d389f5a-9db4-404c-8d4a-7f3f4c73108b","Type":"ContainerDied","Data":"c2fbc4ef4d845eae77fff7321994fb3a21806e51ad430365426bbf2ab02b88c9"}
Jul 26 18:45:55.907769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.907698    2559 scope.go:117] "RemoveContainer" containerID="79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec"
Jul 26 18:45:55.916904 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.916884    2559 scope.go:117] "RemoveContainer" containerID="79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec"
Jul 26 18:45:55.917172 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:45:55.917153    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec\": container with ID starting with 79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec not found: ID does not exist" containerID="79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec"
Jul 26 18:45:55.917217 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.917180    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec"} err="failed to get container status \"79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec\": rpc error: code = NotFound desc = could not find container \"79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec\": container with ID starting with 79c0eb9914132c1a995318abed4ef78244345dfc50929884e108edbc6e29bbec not found: ID does not exist"
Jul 26 18:45:55.932744 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.932716    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"]
Jul 26 18:45:55.936496 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:55.936472    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-1d6ece93/e2e-lifecycle-model-1d6ece93-kserve-686b8d5667-n4m4q"]
Jul 26 18:45:57.177346 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:45:57.177318    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d389f5a-9db4-404c-8d4a-7f3f4c73108b" path="/var/lib/kubelet/pods/4d389f5a-9db4-404c-8d4a-7f3f4c73108b/volumes"
Jul 26 18:47:09.920369 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:09.920343    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:47:09.928049 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:09.928031    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:47:27.406023 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.405991    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"]
Jul 26 18:47:27.406426 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.406359    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261bed6d-b004-4304-bdc7-88438d79bae5" containerName="payload-pre-processing"
Jul 26 18:47:27.406426 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.406370    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="261bed6d-b004-4304-bdc7-88438d79bae5" containerName="payload-pre-processing"
Jul 26 18:47:27.406426 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.406379    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e064a15-60a6-489d-8721-912b7e7264e9" containerName="payload-processing"
Jul 26 18:47:27.406426 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.406384    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e064a15-60a6-489d-8721-912b7e7264e9" containerName="payload-processing"
Jul 26 18:47:27.406426 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.406409    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d389f5a-9db4-404c-8d4a-7f3f4c73108b" containerName="main"
Jul 26 18:47:27.406426 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.406418    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d389f5a-9db4-404c-8d4a-7f3f4c73108b" containerName="main"
Jul 26 18:47:27.406676 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.406519    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="261bed6d-b004-4304-bdc7-88438d79bae5" containerName="payload-pre-processing"
Jul 26 18:47:27.406676 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.406532    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e064a15-60a6-489d-8721-912b7e7264e9" containerName="payload-processing"
Jul 26 18:47:27.406676 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.406546    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d389f5a-9db4-404c-8d4a-7f3f4c73108b" containerName="main"
Jul 26 18:47:27.409008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.408994    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:27.411966 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.411936    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-6329ea\""
Jul 26 18:47:27.412079 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.411982    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-6329ea-dockercfg-8f4jg\""
Jul 26 18:47:27.419220 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.419194    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"]
Jul 26 18:47:27.432350 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.432325    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"]
Jul 26 18:47:27.436329 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.436312    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:27.446089 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.446065    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"]
Jul 26 18:47:27.552464 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.552429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmkmg\" (UniqueName: \"kubernetes.io/projected/128d45fc-7e19-4a54-9abc-195376d5c294-kube-api-access-wmkmg\") pod \"payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7\" (UID: \"128d45fc-7e19-4a54-9abc-195376d5c294\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:27.552650 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.552481    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128d45fc-7e19-4a54-9abc-195376d5c294-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7\" (UID: \"128d45fc-7e19-4a54-9abc-195376d5c294\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:27.552650 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.552572    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn5rh\" (UniqueName: \"kubernetes.io/projected/fbf251c9-9776-4d7e-a14a-4ada9aab7937-kube-api-access-fn5rh\") pod \"payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c\" (UID: \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:27.552650 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.552595    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf251c9-9776-4d7e-a14a-4ada9aab7937-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c\" (UID: \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:27.653415 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.653381    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128d45fc-7e19-4a54-9abc-195376d5c294-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7\" (UID: \"128d45fc-7e19-4a54-9abc-195376d5c294\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:27.653601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.653445    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn5rh\" (UniqueName: \"kubernetes.io/projected/fbf251c9-9776-4d7e-a14a-4ada9aab7937-kube-api-access-fn5rh\") pod \"payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c\" (UID: \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:27.653601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.653485    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf251c9-9776-4d7e-a14a-4ada9aab7937-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c\" (UID: \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:27.653601 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.653544    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmkmg\" (UniqueName: \"kubernetes.io/projected/128d45fc-7e19-4a54-9abc-195376d5c294-kube-api-access-wmkmg\") pod \"payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7\" (UID: \"128d45fc-7e19-4a54-9abc-195376d5c294\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:27.654230 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.654204    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128d45fc-7e19-4a54-9abc-195376d5c294-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7\" (UID: \"128d45fc-7e19-4a54-9abc-195376d5c294\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:27.654331 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.654235    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf251c9-9776-4d7e-a14a-4ada9aab7937-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c\" (UID: \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:27.661725 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.661669    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmkmg\" (UniqueName: \"kubernetes.io/projected/128d45fc-7e19-4a54-9abc-195376d5c294-kube-api-access-wmkmg\") pod \"payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7\" (UID: \"128d45fc-7e19-4a54-9abc-195376d5c294\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:27.661869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.661842    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn5rh\" (UniqueName: \"kubernetes.io/projected/fbf251c9-9776-4d7e-a14a-4ada9aab7937-kube-api-access-fn5rh\") pod \"payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c\" (UID: \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\") " pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:27.720804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.720780    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:27.747961 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.747938    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:27.874489 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.874461    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"]
Jul 26 18:47:27.878037 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:47:27.878005    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128d45fc_7e19_4a54_9abc_195376d5c294.slice/crio-016627d06feb6e3fe16718631cecbfed8c6b8a7806744130a5d213c184677350 WatchSource:0}: Error finding container 016627d06feb6e3fe16718631cecbfed8c6b8a7806744130a5d213c184677350: Status 404 returned error can't find the container with id 016627d06feb6e3fe16718631cecbfed8c6b8a7806744130a5d213c184677350
Jul 26 18:47:27.898704 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:27.898680    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"]
Jul 26 18:47:27.900155 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:47:27.900124    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf251c9_9776_4d7e_a14a_4ada9aab7937.slice/crio-75906082c6903378dccf0f80ba602b7588e36a8342b5c5db80dfaf6eefd88100 WatchSource:0}: Error finding container 75906082c6903378dccf0f80ba602b7588e36a8342b5c5db80dfaf6eefd88100: Status 404 returned error can't find the container with id 75906082c6903378dccf0f80ba602b7588e36a8342b5c5db80dfaf6eefd88100
Jul 26 18:47:28.348428 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:28.348326    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7" event={"ID":"128d45fc-7e19-4a54-9abc-195376d5c294","Type":"ContainerStarted","Data":"56fa01576e04e53510e3650a452378034235fafc7ed7ac40a3c73f150b447f88"}
Jul 26 18:47:28.348428 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:28.348389    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7" event={"ID":"128d45fc-7e19-4a54-9abc-195376d5c294","Type":"ContainerStarted","Data":"016627d06feb6e3fe16718631cecbfed8c6b8a7806744130a5d213c184677350"}
Jul 26 18:47:28.348692 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:28.348448    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:28.350061 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:28.350027    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c" event={"ID":"fbf251c9-9776-4d7e-a14a-4ada9aab7937","Type":"ContainerStarted","Data":"99116477b791701d15ec6d54d1699b029a70c3ff0b2da187885d5195b5543aa3"}
Jul 26 18:47:28.350061 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:28.350067    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c" event={"ID":"fbf251c9-9776-4d7e-a14a-4ada9aab7937","Type":"ContainerStarted","Data":"75906082c6903378dccf0f80ba602b7588e36a8342b5c5db80dfaf6eefd88100"}
Jul 26 18:47:28.350272 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:28.350100    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:28.368359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:28.368314    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7" podStartSLOduration=1.36830068 podStartE2EDuration="1.36830068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:28.365868386 +0000 UTC m=+2119.690390485" watchObservedRunningTime="2026-07-26 18:47:28.36830068 +0000 UTC m=+2119.692822777"
Jul 26 18:47:28.384727 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:28.384685    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c" podStartSLOduration=1.384671629 podStartE2EDuration="1.384671629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:28.383226333 +0000 UTC m=+2119.707748432" watchObservedRunningTime="2026-07-26 18:47:28.384671629 +0000 UTC m=+2119.709193768"
Jul 26 18:47:35.376644 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.376595    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"]
Jul 26 18:47:35.380206 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.380184    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:35.386377 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.386351    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ca4228\""
Jul 26 18:47:35.397508 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.397474    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"]
Jul 26 18:47:35.527768 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.527729    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt82j\" (UniqueName: \"kubernetes.io/projected/62a34493-3890-4ed9-93cd-a6c1026479dc-kube-api-access-lt82j\") pod \"payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264\" (UID: \"62a34493-3890-4ed9-93cd-a6c1026479dc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:35.527925 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.527816    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62a34493-3890-4ed9-93cd-a6c1026479dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264\" (UID: \"62a34493-3890-4ed9-93cd-a6c1026479dc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:35.629001 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.628925    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt82j\" (UniqueName: \"kubernetes.io/projected/62a34493-3890-4ed9-93cd-a6c1026479dc-kube-api-access-lt82j\") pod \"payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264\" (UID: \"62a34493-3890-4ed9-93cd-a6c1026479dc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:35.629139 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.629011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62a34493-3890-4ed9-93cd-a6c1026479dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264\" (UID: \"62a34493-3890-4ed9-93cd-a6c1026479dc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:35.629718 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.629692    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62a34493-3890-4ed9-93cd-a6c1026479dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264\" (UID: \"62a34493-3890-4ed9-93cd-a6c1026479dc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:35.645173 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.639357    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt82j\" (UniqueName: \"kubernetes.io/projected/62a34493-3890-4ed9-93cd-a6c1026479dc-kube-api-access-lt82j\") pod \"payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264\" (UID: \"62a34493-3890-4ed9-93cd-a6c1026479dc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:35.691519 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:35.691489    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:36.035681 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:36.035655    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"]
Jul 26 18:47:36.038489 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:47:36.038457    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62a34493_3890_4ed9_93cd_a6c1026479dc.slice/crio-f46f964188b2d136d8ffc0646e8f58fb7bbe1c5af351c43c97615dbe240bb1ef WatchSource:0}: Error finding container f46f964188b2d136d8ffc0646e8f58fb7bbe1c5af351c43c97615dbe240bb1ef: Status 404 returned error can't find the container with id f46f964188b2d136d8ffc0646e8f58fb7bbe1c5af351c43c97615dbe240bb1ef
Jul 26 18:47:36.392819 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:36.392729    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264" event={"ID":"62a34493-3890-4ed9-93cd-a6c1026479dc","Type":"ContainerStarted","Data":"722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823"}
Jul 26 18:47:36.392819 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:36.392785    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264" event={"ID":"62a34493-3890-4ed9-93cd-a6c1026479dc","Type":"ContainerStarted","Data":"f46f964188b2d136d8ffc0646e8f58fb7bbe1c5af351c43c97615dbe240bb1ef"}
Jul 26 18:47:36.393223 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:36.392871    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:36.409477 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:36.409428    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264" podStartSLOduration=1.409414314 podStartE2EDuration="1.409414314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:36.40791192 +0000 UTC m=+2127.732434017" watchObservedRunningTime="2026-07-26 18:47:36.409414314 +0000 UTC m=+2127.733936439"
Jul 26 18:47:39.357257 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:39.357229    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:47:39.357691 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:39.357293    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:47:47.399966 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:47.399933    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:47:57.097140 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.097105    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"]
Jul 26 18:47:57.100433 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.100413    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.103237 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.103207    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6329ea\"/\"openshift-service-ca.crt\""
Jul 26 18:47:57.104668 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.104521    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6329ea\"/\"kube-root-ca.crt\""
Jul 26 18:47:57.104668 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.104540    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6329ea\"/\"default-dockercfg-pvgzq\""
Jul 26 18:47:57.104668 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.104551    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6329ea\"/\"test-model-6329ea-kserve-self-signed-certs\""
Jul 26 18:47:57.109808 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.109784    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"]
Jul 26 18:47:57.235840 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.235810    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tls-certs\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.236017 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.235851    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-model-cache\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.236017 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.235899    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv84d\" (UniqueName: \"kubernetes.io/projected/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-kube-api-access-tv84d\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.236017 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.235932    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-dshm\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.236017 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.235999    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tmp-dir\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.236174 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.236031    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-home\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.337331 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.337301    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tls-certs\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.337514 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.337349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-model-cache\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.337514 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.337399    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv84d\" (UniqueName: \"kubernetes.io/projected/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-kube-api-access-tv84d\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.337514 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.337442    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-dshm\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.337514 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.337502    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tmp-dir\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.337761 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.337530    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-home\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.338015 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.337981    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tmp-dir\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.338105 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.338015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-home\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.338105 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.338049    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-model-cache\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.339739 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.339713    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-dshm\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.339915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.339894    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tls-certs\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.346191 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.346167    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv84d\" (UniqueName: \"kubernetes.io/projected/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-kube-api-access-tv84d\") pod \"test-model-6329ea-kserve-85bb7df46f-w9srx\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") " pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.411761 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.411679    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:47:57.549749 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:57.549719    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"]
Jul 26 18:47:57.551661 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:47:57.551633    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda625d027_6dbb_4f34_a8f9_64d3f140f6c3.slice/crio-4be9af02b664ade6e4bb56cf60c62b893670f6117d471ae155d3022588b9caa6 WatchSource:0}: Error finding container 4be9af02b664ade6e4bb56cf60c62b893670f6117d471ae155d3022588b9caa6: Status 404 returned error can't find the container with id 4be9af02b664ade6e4bb56cf60c62b893670f6117d471ae155d3022588b9caa6
Jul 26 18:47:58.498214 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:58.498174    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx" event={"ID":"a625d027-6dbb-4f34-a8f9-64d3f140f6c3","Type":"ContainerStarted","Data":"99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871"}
Jul 26 18:47:58.498583 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:58.498228    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx" event={"ID":"a625d027-6dbb-4f34-a8f9-64d3f140f6c3","Type":"ContainerStarted","Data":"4be9af02b664ade6e4bb56cf60c62b893670f6117d471ae155d3022588b9caa6"}
Jul 26 18:47:58.516895 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:47:58.516843    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx" podStartSLOduration=1.5168265490000001 podStartE2EDuration="1.516826549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:47:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:47:58.516087865 +0000 UTC m=+2149.840609963" watchObservedRunningTime="2026-07-26 18:47:58.516826549 +0000 UTC m=+2149.841348646"
Jul 26 18:48:07.412097 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:07.412018    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:48:07.412469 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:07.412230    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:48:07.425028 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:07.425003    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:48:07.553796 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:07.553769    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:48:32.303530 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.303496    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"]
Jul 26 18:48:32.307444 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.307426    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.311723 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.311701    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ca4228\"/\"kube-root-ca.crt\""
Jul 26 18:48:32.311869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.311738    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ca4228\"/\"test-model-ca4228-kserve-self-signed-certs\""
Jul 26 18:48:32.311869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.311784    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-ca4228\"/\"default-dockercfg-th87d\""
Jul 26 18:48:32.311869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.311799    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-ca4228\"/\"openshift-service-ca.crt\""
Jul 26 18:48:32.316804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.316782    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"]
Jul 26 18:48:32.366248 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.366212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e258500-108b-4b14-a87f-d662b94a6c4c-tls-certs\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.366401 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.366284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-dshm\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.366401 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.366320    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-home\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.366401 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.366338    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnhgp\" (UniqueName: \"kubernetes.io/projected/1e258500-108b-4b14-a87f-d662b94a6c4c-kube-api-access-wnhgp\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.366577 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.366476    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-tmp-dir\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.366923 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.366523    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-model-cache\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.467476 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.467428    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e258500-108b-4b14-a87f-d662b94a6c4c-tls-certs\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.467675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.467496    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-dshm\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.467675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.467532    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-home\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.467675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.467554    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnhgp\" (UniqueName: \"kubernetes.io/projected/1e258500-108b-4b14-a87f-d662b94a6c4c-kube-api-access-wnhgp\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.467675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.467597    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-tmp-dir\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.467675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.467653    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-model-cache\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.468021 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.467997    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-home\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.468021 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.468015    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-tmp-dir\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.468100 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.468087    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-model-cache\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.469804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.469777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-dshm\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.469933 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.469914    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e258500-108b-4b14-a87f-d662b94a6c4c-tls-certs\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.475610 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.475585    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnhgp\" (UniqueName: \"kubernetes.io/projected/1e258500-108b-4b14-a87f-d662b94a6c4c-kube-api-access-wnhgp\") pod \"test-model-ca4228-kserve-74c667854f-bgtt5\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") " pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.617674 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.617593    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:32.752536 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:32.752505    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"]
Jul 26 18:48:32.755470 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:48:32.755437    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e258500_108b_4b14_a87f_d662b94a6c4c.slice/crio-587398d54cc9a1c90f80c46588674db773481f5f4af6efe100d312e8e8c74d9a WatchSource:0}: Error finding container 587398d54cc9a1c90f80c46588674db773481f5f4af6efe100d312e8e8c74d9a: Status 404 returned error can't find the container with id 587398d54cc9a1c90f80c46588674db773481f5f4af6efe100d312e8e8c74d9a
Jul 26 18:48:33.663904 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:33.663861    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5" event={"ID":"1e258500-108b-4b14-a87f-d662b94a6c4c","Type":"ContainerStarted","Data":"0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88"}
Jul 26 18:48:33.663904 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:33.663906    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5" event={"ID":"1e258500-108b-4b14-a87f-d662b94a6c4c","Type":"ContainerStarted","Data":"587398d54cc9a1c90f80c46588674db773481f5f4af6efe100d312e8e8c74d9a"}
Jul 26 18:48:33.682080 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:33.682030    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5" podStartSLOduration=1.6820160240000002 podStartE2EDuration="1.682016024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:48:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:48:33.681278395 +0000 UTC m=+2185.005800493" watchObservedRunningTime="2026-07-26 18:48:33.682016024 +0000 UTC m=+2185.006538122"
Jul 26 18:48:42.618005 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:42.617969    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:42.618005 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:42.618013    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:42.630576 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:42.630552    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:48:42.718843 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:48:42.718817    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:49:14.255816 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:14.255587    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"]
Jul 26 18:49:15.219500 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:15.219466    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"]
Jul 26 18:49:25.776842 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:25.776810    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"]
Jul 26 18:49:25.777304 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:25.777087    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c" podUID="fbf251c9-9776-4d7e-a14a-4ada9aab7937" containerName="payload-processing" containerID="cri-o://99116477b791701d15ec6d54d1699b029a70c3ff0b2da187885d5195b5543aa3" gracePeriod=30
Jul 26 18:49:25.782954 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:25.782925    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"]
Jul 26 18:49:25.783317 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:25.783241    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7" podUID="128d45fc-7e19-4a54-9abc-195376d5c294" containerName="payload-pre-processing" containerID="cri-o://56fa01576e04e53510e3650a452378034235fafc7ed7ac40a3c73f150b447f88" gracePeriod=30
Jul 26 18:49:25.932180 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:25.932143    2559 generic.go:356] "Generic (PLEG): container finished" podID="fbf251c9-9776-4d7e-a14a-4ada9aab7937" containerID="99116477b791701d15ec6d54d1699b029a70c3ff0b2da187885d5195b5543aa3" exitCode=0
Jul 26 18:49:25.932367 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:25.932304    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c" event={"ID":"fbf251c9-9776-4d7e-a14a-4ada9aab7937","Type":"ContainerDied","Data":"99116477b791701d15ec6d54d1699b029a70c3ff0b2da187885d5195b5543aa3"}
Jul 26 18:49:25.934581 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:25.934319    2559 generic.go:356] "Generic (PLEG): container finished" podID="128d45fc-7e19-4a54-9abc-195376d5c294" containerID="56fa01576e04e53510e3650a452378034235fafc7ed7ac40a3c73f150b447f88" exitCode=0
Jul 26 18:49:25.934581 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:25.934361    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7" event={"ID":"128d45fc-7e19-4a54-9abc-195376d5c294","Type":"ContainerDied","Data":"56fa01576e04e53510e3650a452378034235fafc7ed7ac40a3c73f150b447f88"}
Jul 26 18:49:26.044910 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.044852    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:49:26.066831 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.066807    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:49:26.168954 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.168928    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf251c9-9776-4d7e-a14a-4ada9aab7937-plugins-config-volume\") pod \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\" (UID: \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\") "
Jul 26 18:49:26.169111 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.168988    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128d45fc-7e19-4a54-9abc-195376d5c294-plugins-config-volume\") pod \"128d45fc-7e19-4a54-9abc-195376d5c294\" (UID: \"128d45fc-7e19-4a54-9abc-195376d5c294\") "
Jul 26 18:49:26.169111 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.169057    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmkmg\" (UniqueName: \"kubernetes.io/projected/128d45fc-7e19-4a54-9abc-195376d5c294-kube-api-access-wmkmg\") pod \"128d45fc-7e19-4a54-9abc-195376d5c294\" (UID: \"128d45fc-7e19-4a54-9abc-195376d5c294\") "
Jul 26 18:49:26.169111 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.169092    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn5rh\" (UniqueName: \"kubernetes.io/projected/fbf251c9-9776-4d7e-a14a-4ada9aab7937-kube-api-access-fn5rh\") pod \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\" (UID: \"fbf251c9-9776-4d7e-a14a-4ada9aab7937\") "
Jul 26 18:49:26.169404 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.169377    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf251c9-9776-4d7e-a14a-4ada9aab7937-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fbf251c9-9776-4d7e-a14a-4ada9aab7937" (UID: "fbf251c9-9776-4d7e-a14a-4ada9aab7937"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:26.169471 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.169377    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128d45fc-7e19-4a54-9abc-195376d5c294-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "128d45fc-7e19-4a54-9abc-195376d5c294" (UID: "128d45fc-7e19-4a54-9abc-195376d5c294"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:26.171166 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.171145    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/128d45fc-7e19-4a54-9abc-195376d5c294-kube-api-access-wmkmg" (OuterVolumeSpecName: "kube-api-access-wmkmg") pod "128d45fc-7e19-4a54-9abc-195376d5c294" (UID: "128d45fc-7e19-4a54-9abc-195376d5c294"). InnerVolumeSpecName "kube-api-access-wmkmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:26.171277 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.171260    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbf251c9-9776-4d7e-a14a-4ada9aab7937-kube-api-access-fn5rh" (OuterVolumeSpecName: "kube-api-access-fn5rh") pod "fbf251c9-9776-4d7e-a14a-4ada9aab7937" (UID: "fbf251c9-9776-4d7e-a14a-4ada9aab7937"). InnerVolumeSpecName "kube-api-access-fn5rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:26.270451 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.270421    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn5rh\" (UniqueName: \"kubernetes.io/projected/fbf251c9-9776-4d7e-a14a-4ada9aab7937-kube-api-access-fn5rh\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:26.270451 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.270449    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbf251c9-9776-4d7e-a14a-4ada9aab7937-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:26.270628 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.270462    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/128d45fc-7e19-4a54-9abc-195376d5c294-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:26.270628 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.270473    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmkmg\" (UniqueName: \"kubernetes.io/projected/128d45fc-7e19-4a54-9abc-195376d5c294-kube-api-access-wmkmg\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:26.940684 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.940654    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"
Jul 26 18:49:26.941152 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.940658    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7" event={"ID":"128d45fc-7e19-4a54-9abc-195376d5c294","Type":"ContainerDied","Data":"016627d06feb6e3fe16718631cecbfed8c6b8a7806744130a5d213c184677350"}
Jul 26 18:49:26.941152 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.940788    2559 scope.go:117] "RemoveContainer" containerID="56fa01576e04e53510e3650a452378034235fafc7ed7ac40a3c73f150b447f88"
Jul 26 18:49:26.942205 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.942109    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c" event={"ID":"fbf251c9-9776-4d7e-a14a-4ada9aab7937","Type":"ContainerDied","Data":"75906082c6903378dccf0f80ba602b7588e36a8342b5c5db80dfaf6eefd88100"}
Jul 26 18:49:26.942205 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.942120    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"
Jul 26 18:49:26.950769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.950752    2559 scope.go:117] "RemoveContainer" containerID="99116477b791701d15ec6d54d1699b029a70c3ff0b2da187885d5195b5543aa3"
Jul 26 18:49:26.963589 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.963567    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"]
Jul 26 18:49:26.966748 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.966727    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-6329ea-566c6fc496-jr7z7"]
Jul 26 18:49:26.977246 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.977223    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"]
Jul 26 18:49:26.980600 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:26.980577    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-6329ea-85bb97d86-mt62c"]
Jul 26 18:49:27.177470 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:27.177440    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="128d45fc-7e19-4a54-9abc-195376d5c294" path="/var/lib/kubelet/pods/128d45fc-7e19-4a54-9abc-195376d5c294/volumes"
Jul 26 18:49:27.177827 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:27.177812    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbf251c9-9776-4d7e-a14a-4ada9aab7937" path="/var/lib/kubelet/pods/fbf251c9-9776-4d7e-a14a-4ada9aab7937/volumes"
Jul 26 18:49:29.278414 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.278377    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx" podUID="a625d027-6dbb-4f34-a8f9-64d3f140f6c3" containerName="main" containerID="cri-o://99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871" gracePeriod=45
Jul 26 18:49:29.528368 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.528343    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:49:29.703546 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.703515    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tls-certs\") pod \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") "
Jul 26 18:49:29.703733 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.703556    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-dshm\") pod \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") "
Jul 26 18:49:29.703733 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.703580    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-home\") pod \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") "
Jul 26 18:49:29.703733 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.703625    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tv84d\" (UniqueName: \"kubernetes.io/projected/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-kube-api-access-tv84d\") pod \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") "
Jul 26 18:49:29.703733 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.703655    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-model-cache\") pod \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") "
Jul 26 18:49:29.703733 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.703690    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tmp-dir\") pod \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\" (UID: \"a625d027-6dbb-4f34-a8f9-64d3f140f6c3\") "
Jul 26 18:49:29.704033 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.703869    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-home" (OuterVolumeSpecName: "home") pod "a625d027-6dbb-4f34-a8f9-64d3f140f6c3" (UID: "a625d027-6dbb-4f34-a8f9-64d3f140f6c3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:29.704107 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.704081    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a625d027-6dbb-4f34-a8f9-64d3f140f6c3" (UID: "a625d027-6dbb-4f34-a8f9-64d3f140f6c3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:29.704160 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.704116    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:29.704235 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.704159    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-model-cache" (OuterVolumeSpecName: "model-cache") pod "a625d027-6dbb-4f34-a8f9-64d3f140f6c3" (UID: "a625d027-6dbb-4f34-a8f9-64d3f140f6c3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:29.706012 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.705982    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-dshm" (OuterVolumeSpecName: "dshm") pod "a625d027-6dbb-4f34-a8f9-64d3f140f6c3" (UID: "a625d027-6dbb-4f34-a8f9-64d3f140f6c3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:29.706123 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.706008    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-kube-api-access-tv84d" (OuterVolumeSpecName: "kube-api-access-tv84d") pod "a625d027-6dbb-4f34-a8f9-64d3f140f6c3" (UID: "a625d027-6dbb-4f34-a8f9-64d3f140f6c3"). InnerVolumeSpecName "kube-api-access-tv84d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:29.706123 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.705986    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a625d027-6dbb-4f34-a8f9-64d3f140f6c3" (UID: "a625d027-6dbb-4f34-a8f9-64d3f140f6c3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:49:29.804654 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.804605    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv84d\" (UniqueName: \"kubernetes.io/projected/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-kube-api-access-tv84d\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:29.804654 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.804651    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:29.804852 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.804664    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:29.804852 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.804675    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:29.804852 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.804686    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a625d027-6dbb-4f34-a8f9-64d3f140f6c3-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:29.959707 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.959596    2559 generic.go:356] "Generic (PLEG): container finished" podID="a625d027-6dbb-4f34-a8f9-64d3f140f6c3" containerID="99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871" exitCode=0
Jul 26 18:49:29.959707 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.959667    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"
Jul 26 18:49:29.959707 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.959678    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx" event={"ID":"a625d027-6dbb-4f34-a8f9-64d3f140f6c3","Type":"ContainerDied","Data":"99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871"}
Jul 26 18:49:29.959930 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.959724    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx" event={"ID":"a625d027-6dbb-4f34-a8f9-64d3f140f6c3","Type":"ContainerDied","Data":"4be9af02b664ade6e4bb56cf60c62b893670f6117d471ae155d3022588b9caa6"}
Jul 26 18:49:29.959930 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.959743    2559 scope.go:117] "RemoveContainer" containerID="99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871"
Jul 26 18:49:29.968685 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.968653    2559 scope.go:117] "RemoveContainer" containerID="99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871"
Jul 26 18:49:29.968927 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:49:29.968907    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871\": container with ID starting with 99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871 not found: ID does not exist" containerID="99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871"
Jul 26 18:49:29.968976 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.968935    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871"} err="failed to get container status \"99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871\": rpc error: code = NotFound desc = could not find container \"99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871\": container with ID starting with 99dccbc1c8b945c3c4b2ef0988157d876ea295e3fec9b73060a7c93d71b08871 not found: ID does not exist"
Jul 26 18:49:29.981166 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.981143    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"]
Jul 26 18:49:29.985478 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:29.985459    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6329ea/test-model-6329ea-kserve-85bb7df46f-w9srx"]
Jul 26 18:49:30.239328 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.239251    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5" podUID="1e258500-108b-4b14-a87f-d662b94a6c4c" containerName="main" containerID="cri-o://0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88" gracePeriod=45
Jul 26 18:49:30.493996 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.493935    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:49:30.612405 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612342    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-tmp-dir\") pod \"1e258500-108b-4b14-a87f-d662b94a6c4c\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") "
Jul 26 18:49:30.612405 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612383    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-model-cache\") pod \"1e258500-108b-4b14-a87f-d662b94a6c4c\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") "
Jul 26 18:49:30.612641 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612423    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e258500-108b-4b14-a87f-d662b94a6c4c-tls-certs\") pod \"1e258500-108b-4b14-a87f-d662b94a6c4c\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") "
Jul 26 18:49:30.612641 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612449    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-home\") pod \"1e258500-108b-4b14-a87f-d662b94a6c4c\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") "
Jul 26 18:49:30.612641 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612525    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnhgp\" (UniqueName: \"kubernetes.io/projected/1e258500-108b-4b14-a87f-d662b94a6c4c-kube-api-access-wnhgp\") pod \"1e258500-108b-4b14-a87f-d662b94a6c4c\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") "
Jul 26 18:49:30.612641 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612562    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-dshm\") pod \"1e258500-108b-4b14-a87f-d662b94a6c4c\" (UID: \"1e258500-108b-4b14-a87f-d662b94a6c4c\") "
Jul 26 18:49:30.612868 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612640    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1e258500-108b-4b14-a87f-d662b94a6c4c" (UID: "1e258500-108b-4b14-a87f-d662b94a6c4c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:30.612868 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612681    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-model-cache" (OuterVolumeSpecName: "model-cache") pod "1e258500-108b-4b14-a87f-d662b94a6c4c" (UID: "1e258500-108b-4b14-a87f-d662b94a6c4c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:30.612868 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612776    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-home" (OuterVolumeSpecName: "home") pod "1e258500-108b-4b14-a87f-d662b94a6c4c" (UID: "1e258500-108b-4b14-a87f-d662b94a6c4c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:30.612977 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612881    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:30.612977 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612894    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:30.612977 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.612902    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:30.614549 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.614529    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e258500-108b-4b14-a87f-d662b94a6c4c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1e258500-108b-4b14-a87f-d662b94a6c4c" (UID: "1e258500-108b-4b14-a87f-d662b94a6c4c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:49:30.614637 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.614582    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-dshm" (OuterVolumeSpecName: "dshm") pod "1e258500-108b-4b14-a87f-d662b94a6c4c" (UID: "1e258500-108b-4b14-a87f-d662b94a6c4c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:49:30.614742 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.614723    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e258500-108b-4b14-a87f-d662b94a6c4c-kube-api-access-wnhgp" (OuterVolumeSpecName: "kube-api-access-wnhgp") pod "1e258500-108b-4b14-a87f-d662b94a6c4c" (UID: "1e258500-108b-4b14-a87f-d662b94a6c4c"). InnerVolumeSpecName "kube-api-access-wnhgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:30.713743 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.713713    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e258500-108b-4b14-a87f-d662b94a6c4c-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:30.713743 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.713740    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e258500-108b-4b14-a87f-d662b94a6c4c-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:30.713894 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.713752    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnhgp\" (UniqueName: \"kubernetes.io/projected/1e258500-108b-4b14-a87f-d662b94a6c4c-kube-api-access-wnhgp\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:30.966018 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.965985    2559 generic.go:356] "Generic (PLEG): container finished" podID="1e258500-108b-4b14-a87f-d662b94a6c4c" containerID="0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88" exitCode=0
Jul 26 18:49:30.966176 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.966031    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"
Jul 26 18:49:30.966176 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.966053    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5" event={"ID":"1e258500-108b-4b14-a87f-d662b94a6c4c","Type":"ContainerDied","Data":"0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88"}
Jul 26 18:49:30.966176 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.966084    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5" event={"ID":"1e258500-108b-4b14-a87f-d662b94a6c4c","Type":"ContainerDied","Data":"587398d54cc9a1c90f80c46588674db773481f5f4af6efe100d312e8e8c74d9a"}
Jul 26 18:49:30.966176 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.966102    2559 scope.go:117] "RemoveContainer" containerID="0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88"
Jul 26 18:49:30.974865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.974846    2559 scope.go:117] "RemoveContainer" containerID="0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88"
Jul 26 18:49:30.975089 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:49:30.975069    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88\": container with ID starting with 0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88 not found: ID does not exist" containerID="0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88"
Jul 26 18:49:30.975147 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.975097    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88"} err="failed to get container status \"0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88\": rpc error: code = NotFound desc = could not find container \"0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88\": container with ID starting with 0a20adb1892f1552528f6d5958117ab7dac7bec30a2e89c800ef7798d8185c88 not found: ID does not exist"
Jul 26 18:49:30.990600 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.990577    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"]
Jul 26 18:49:30.998981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:30.998961    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-ca4228/test-model-ca4228-kserve-74c667854f-bgtt5"]
Jul 26 18:49:31.176319 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:31.176288    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e258500-108b-4b14-a87f-d662b94a6c4c" path="/var/lib/kubelet/pods/1e258500-108b-4b14-a87f-d662b94a6c4c/volumes"
Jul 26 18:49:31.176676 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:31.176662    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a625d027-6dbb-4f34-a8f9-64d3f140f6c3" path="/var/lib/kubelet/pods/a625d027-6dbb-4f34-a8f9-64d3f140f6c3/volumes"
Jul 26 18:49:37.099373 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099328    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"]
Jul 26 18:49:37.099871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099770    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e258500-108b-4b14-a87f-d662b94a6c4c" containerName="main"
Jul 26 18:49:37.099871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099785    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e258500-108b-4b14-a87f-d662b94a6c4c" containerName="main"
Jul 26 18:49:37.099871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099798    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf251c9-9776-4d7e-a14a-4ada9aab7937" containerName="payload-processing"
Jul 26 18:49:37.099871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099803    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf251c9-9776-4d7e-a14a-4ada9aab7937" containerName="payload-processing"
Jul 26 18:49:37.099871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099810    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a625d027-6dbb-4f34-a8f9-64d3f140f6c3" containerName="main"
Jul 26 18:49:37.099871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099815    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a625d027-6dbb-4f34-a8f9-64d3f140f6c3" containerName="main"
Jul 26 18:49:37.099871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099827    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="128d45fc-7e19-4a54-9abc-195376d5c294" containerName="payload-pre-processing"
Jul 26 18:49:37.099871 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099832    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="128d45fc-7e19-4a54-9abc-195376d5c294" containerName="payload-pre-processing"
Jul 26 18:49:37.100298 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099906    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a625d027-6dbb-4f34-a8f9-64d3f140f6c3" containerName="main"
Jul 26 18:49:37.100298 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099919    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="128d45fc-7e19-4a54-9abc-195376d5c294" containerName="payload-pre-processing"
Jul 26 18:49:37.100298 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099928    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf251c9-9776-4d7e-a14a-4ada9aab7937" containerName="payload-processing"
Jul 26 18:49:37.100298 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.099935    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e258500-108b-4b14-a87f-d662b94a6c4c" containerName="main"
Jul 26 18:49:37.104451 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.104426    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"
Jul 26 18:49:37.107349 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.107313    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:49:37.107474 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.107420    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 18:49:37.111386 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.111362    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"]
Jul 26 18:49:37.167900 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.167860    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3333ad44-5f0b-40f5-b897-bec9d9c17ffd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt\" (UID: \"3333ad44-5f0b-40f5-b897-bec9d9c17ffd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"
Jul 26 18:49:37.268585 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.268549    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3333ad44-5f0b-40f5-b897-bec9d9c17ffd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt\" (UID: \"3333ad44-5f0b-40f5-b897-bec9d9c17ffd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"
Jul 26 18:49:37.269165 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.269142    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3333ad44-5f0b-40f5-b897-bec9d9c17ffd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt\" (UID: \"3333ad44-5f0b-40f5-b897-bec9d9c17ffd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"
Jul 26 18:49:37.417059 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.417027    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"
Jul 26 18:49:37.553457 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.553432    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"]
Jul 26 18:49:37.555912 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:49:37.555874    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3333ad44_5f0b_40f5_b897_bec9d9c17ffd.slice/crio-f3674221781b10a6f8fcabdaef50ade988f8bd4f23b366ead529e34f507633a1 WatchSource:0}: Error finding container f3674221781b10a6f8fcabdaef50ade988f8bd4f23b366ead529e34f507633a1: Status 404 returned error can't find the container with id f3674221781b10a6f8fcabdaef50ade988f8bd4f23b366ead529e34f507633a1
Jul 26 18:49:37.557771 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:37.557754    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 18:49:38.005211 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:38.005175    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt" event={"ID":"3333ad44-5f0b-40f5-b897-bec9d9c17ffd","Type":"ContainerStarted","Data":"f3674221781b10a6f8fcabdaef50ade988f8bd4f23b366ead529e34f507633a1"}
Jul 26 18:49:41.021840 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:41.021794    2559 generic.go:356] "Generic (PLEG): container finished" podID="3333ad44-5f0b-40f5-b897-bec9d9c17ffd" containerID="cd19da57af09a32c71025bc9a5f80d8f1a0f5eccb549e3a37333633ac874a42a" exitCode=0
Jul 26 18:49:41.022284 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:41.021869    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt" event={"ID":"3333ad44-5f0b-40f5-b897-bec9d9c17ffd","Type":"ContainerDied","Data":"cd19da57af09a32c71025bc9a5f80d8f1a0f5eccb549e3a37333633ac874a42a"}
Jul 26 18:49:42.160891 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:42.160867    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"
Jul 26 18:49:42.217529 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:42.217495    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3333ad44-5f0b-40f5-b897-bec9d9c17ffd-maas-api-service-ca\") pod \"3333ad44-5f0b-40f5-b897-bec9d9c17ffd\" (UID: \"3333ad44-5f0b-40f5-b897-bec9d9c17ffd\") "
Jul 26 18:49:42.217896 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:42.217864    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3333ad44-5f0b-40f5-b897-bec9d9c17ffd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3333ad44-5f0b-40f5-b897-bec9d9c17ffd" (UID: "3333ad44-5f0b-40f5-b897-bec9d9c17ffd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:42.319156 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:42.319075    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3333ad44-5f0b-40f5-b897-bec9d9c17ffd-maas-api-service-ca\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:43.033550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:43.033521    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"
Jul 26 18:49:43.033765 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:43.033518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt" event={"ID":"3333ad44-5f0b-40f5-b897-bec9d9c17ffd","Type":"ContainerDied","Data":"f3674221781b10a6f8fcabdaef50ade988f8bd4f23b366ead529e34f507633a1"}
Jul 26 18:49:43.033765 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:43.033656    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3674221781b10a6f8fcabdaef50ade988f8bd4f23b366ead529e34f507633a1"
Jul 26 18:49:47.701675 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:47.701632    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"]
Jul 26 18:49:47.702332 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:47.701849    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264" podUID="62a34493-3890-4ed9-93cd-a6c1026479dc" containerName="payload-pre-processing" containerID="cri-o://722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823" gracePeriod=30
Jul 26 18:49:47.963814 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:47.963759    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:49:48.062940 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.062901    2559 generic.go:356] "Generic (PLEG): container finished" podID="62a34493-3890-4ed9-93cd-a6c1026479dc" containerID="722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823" exitCode=0
Jul 26 18:49:48.063118 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.062943    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"
Jul 26 18:49:48.063118 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.062993    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264" event={"ID":"62a34493-3890-4ed9-93cd-a6c1026479dc","Type":"ContainerDied","Data":"722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823"}
Jul 26 18:49:48.063118 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.063053    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264" event={"ID":"62a34493-3890-4ed9-93cd-a6c1026479dc","Type":"ContainerDied","Data":"f46f964188b2d136d8ffc0646e8f58fb7bbe1c5af351c43c97615dbe240bb1ef"}
Jul 26 18:49:48.063118 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.063083    2559 scope.go:117] "RemoveContainer" containerID="722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823"
Jul 26 18:49:48.071200 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.071172    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt82j\" (UniqueName: \"kubernetes.io/projected/62a34493-3890-4ed9-93cd-a6c1026479dc-kube-api-access-lt82j\") pod \"62a34493-3890-4ed9-93cd-a6c1026479dc\" (UID: \"62a34493-3890-4ed9-93cd-a6c1026479dc\") "
Jul 26 18:49:48.071341 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.071210    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62a34493-3890-4ed9-93cd-a6c1026479dc-plugins-config-volume\") pod \"62a34493-3890-4ed9-93cd-a6c1026479dc\" (UID: \"62a34493-3890-4ed9-93cd-a6c1026479dc\") "
Jul 26 18:49:48.071608 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.071587    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62a34493-3890-4ed9-93cd-a6c1026479dc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "62a34493-3890-4ed9-93cd-a6c1026479dc" (UID: "62a34493-3890-4ed9-93cd-a6c1026479dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:49:48.071804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.071775    2559 scope.go:117] "RemoveContainer" containerID="722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823"
Jul 26 18:49:48.072031 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:49:48.072012    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823\": container with ID starting with 722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823 not found: ID does not exist" containerID="722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823"
Jul 26 18:49:48.072106 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.072036    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823"} err="failed to get container status \"722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823\": rpc error: code = NotFound desc = could not find container \"722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823\": container with ID starting with 722bab207e13d7713ccd0f3ac93086e95bffd1e3b2de2299236f949914022823 not found: ID does not exist"
Jul 26 18:49:48.073258 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.073232    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62a34493-3890-4ed9-93cd-a6c1026479dc-kube-api-access-lt82j" (OuterVolumeSpecName: "kube-api-access-lt82j") pod "62a34493-3890-4ed9-93cd-a6c1026479dc" (UID: "62a34493-3890-4ed9-93cd-a6c1026479dc"). InnerVolumeSpecName "kube-api-access-lt82j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:49:48.172552 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.172526    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt82j\" (UniqueName: \"kubernetes.io/projected/62a34493-3890-4ed9-93cd-a6c1026479dc-kube-api-access-lt82j\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:48.172552 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.172550    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62a34493-3890-4ed9-93cd-a6c1026479dc-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:49:48.385532 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.385501    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"]
Jul 26 18:49:48.389749 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:48.389724    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ca4228-79fd9cf56d-6x264"]
Jul 26 18:49:49.176186 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:49.176152    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62a34493-3890-4ed9-93cd-a6c1026479dc" path="/var/lib/kubelet/pods/62a34493-3890-4ed9-93cd-a6c1026479dc/volumes"
Jul 26 18:49:52.177199 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:52.177166    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"]
Jul 26 18:49:52.179245 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:52.179225    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ca4228-4ppgt"]
Jul 26 18:49:53.176317 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:49:53.176284    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3333ad44-5f0b-40f5-b897-bec9d9c17ffd" path="/var/lib/kubelet/pods/3333ad44-5f0b-40f5-b897-bec9d9c17ffd/volumes"
Jul 26 18:50:03.061024 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.060988    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"]
Jul 26 18:50:03.061431 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.061355    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62a34493-3890-4ed9-93cd-a6c1026479dc" containerName="payload-pre-processing"
Jul 26 18:50:03.061431 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.061366    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="62a34493-3890-4ed9-93cd-a6c1026479dc" containerName="payload-pre-processing"
Jul 26 18:50:03.061431 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.061382    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3333ad44-5f0b-40f5-b897-bec9d9c17ffd" containerName="revoke-keys"
Jul 26 18:50:03.061431 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.061387    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3333ad44-5f0b-40f5-b897-bec9d9c17ffd" containerName="revoke-keys"
Jul 26 18:50:03.061650 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.061446    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="62a34493-3890-4ed9-93cd-a6c1026479dc" containerName="payload-pre-processing"
Jul 26 18:50:03.061650 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.061454    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3333ad44-5f0b-40f5-b897-bec9d9c17ffd" containerName="revoke-keys"
Jul 26 18:50:03.066160 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.066142    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:03.069111 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.069091    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-991a15\""
Jul 26 18:50:03.069196 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.069107    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-991a15-dockercfg-s2scz\""
Jul 26 18:50:03.076044 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.076022    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"]
Jul 26 18:50:03.090285 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.090258    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"]
Jul 26 18:50:03.093732 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.093712    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:03.108341 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.108321    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"]
Jul 26 18:50:03.209299 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.209265    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23142c4-6fea-4b66-aee6-956629b7c268-plugins-config-volume\") pod \"payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9\" (UID: \"c23142c4-6fea-4b66-aee6-956629b7c268\") " pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:03.209299 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.209302    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg8gk\" (UniqueName: \"kubernetes.io/projected/24b401e9-c07c-40d0-930e-dfb4a854032c-kube-api-access-fg8gk\") pod \"payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl\" (UID: \"24b401e9-c07c-40d0-930e-dfb4a854032c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:03.209550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.209350    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b401e9-c07c-40d0-930e-dfb4a854032c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl\" (UID: \"24b401e9-c07c-40d0-930e-dfb4a854032c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:03.209550 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.209437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-824gd\" (UniqueName: \"kubernetes.io/projected/c23142c4-6fea-4b66-aee6-956629b7c268-kube-api-access-824gd\") pod \"payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9\" (UID: \"c23142c4-6fea-4b66-aee6-956629b7c268\") " pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:03.310807 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.310768    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23142c4-6fea-4b66-aee6-956629b7c268-plugins-config-volume\") pod \"payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9\" (UID: \"c23142c4-6fea-4b66-aee6-956629b7c268\") " pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:03.310807 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.310809    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg8gk\" (UniqueName: \"kubernetes.io/projected/24b401e9-c07c-40d0-930e-dfb4a854032c-kube-api-access-fg8gk\") pod \"payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl\" (UID: \"24b401e9-c07c-40d0-930e-dfb4a854032c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:03.311012 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.310839    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b401e9-c07c-40d0-930e-dfb4a854032c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl\" (UID: \"24b401e9-c07c-40d0-930e-dfb4a854032c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:03.311012 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.310873    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-824gd\" (UniqueName: \"kubernetes.io/projected/c23142c4-6fea-4b66-aee6-956629b7c268-kube-api-access-824gd\") pod \"payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9\" (UID: \"c23142c4-6fea-4b66-aee6-956629b7c268\") " pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:03.311594 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.311552    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b401e9-c07c-40d0-930e-dfb4a854032c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl\" (UID: \"24b401e9-c07c-40d0-930e-dfb4a854032c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:03.311594 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.311565    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23142c4-6fea-4b66-aee6-956629b7c268-plugins-config-volume\") pod \"payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9\" (UID: \"c23142c4-6fea-4b66-aee6-956629b7c268\") " pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:03.319398 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.319377    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg8gk\" (UniqueName: \"kubernetes.io/projected/24b401e9-c07c-40d0-930e-dfb4a854032c-kube-api-access-fg8gk\") pod \"payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl\" (UID: \"24b401e9-c07c-40d0-930e-dfb4a854032c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:03.319531 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.319414    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-824gd\" (UniqueName: \"kubernetes.io/projected/c23142c4-6fea-4b66-aee6-956629b7c268-kube-api-access-824gd\") pod \"payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9\" (UID: \"c23142c4-6fea-4b66-aee6-956629b7c268\") " pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:03.377760 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.377729    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:03.403835 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.403812    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:03.528431 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.528385    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"]
Jul 26 18:50:03.530447 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:50:03.530414    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b401e9_c07c_40d0_930e_dfb4a854032c.slice/crio-9bf3d3a446ca718e10cd31814c3aa4dd10c94fa8e9d8f1d263413d8b9c2c62a2 WatchSource:0}: Error finding container 9bf3d3a446ca718e10cd31814c3aa4dd10c94fa8e9d8f1d263413d8b9c2c62a2: Status 404 returned error can't find the container with id 9bf3d3a446ca718e10cd31814c3aa4dd10c94fa8e9d8f1d263413d8b9c2c62a2
Jul 26 18:50:03.562049 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:03.562018    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"]
Jul 26 18:50:03.564381 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:50:03.564360    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23142c4_6fea_4b66_aee6_956629b7c268.slice/crio-066233a6b784a1060cb467b269eb2e83a05b6b9ba1ca6f05e7206a6d2ccaa607 WatchSource:0}: Error finding container 066233a6b784a1060cb467b269eb2e83a05b6b9ba1ca6f05e7206a6d2ccaa607: Status 404 returned error can't find the container with id 066233a6b784a1060cb467b269eb2e83a05b6b9ba1ca6f05e7206a6d2ccaa607
Jul 26 18:50:04.141253 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:04.141207    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9" event={"ID":"c23142c4-6fea-4b66-aee6-956629b7c268","Type":"ContainerStarted","Data":"b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f"}
Jul 26 18:50:04.141760 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:04.141277    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:04.141760 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:04.141301    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9" event={"ID":"c23142c4-6fea-4b66-aee6-956629b7c268","Type":"ContainerStarted","Data":"066233a6b784a1060cb467b269eb2e83a05b6b9ba1ca6f05e7206a6d2ccaa607"}
Jul 26 18:50:04.142997 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:04.142960    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl" event={"ID":"24b401e9-c07c-40d0-930e-dfb4a854032c","Type":"ContainerStarted","Data":"d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7"}
Jul 26 18:50:04.143140 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:04.143007    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl" event={"ID":"24b401e9-c07c-40d0-930e-dfb4a854032c","Type":"ContainerStarted","Data":"9bf3d3a446ca718e10cd31814c3aa4dd10c94fa8e9d8f1d263413d8b9c2c62a2"}
Jul 26 18:50:04.143140 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:04.143094    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:04.159008 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:04.158961    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9" podStartSLOduration=1.158948524 podStartE2EDuration="1.158948524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:04.157573599 +0000 UTC m=+2275.482095718" watchObservedRunningTime="2026-07-26 18:50:04.158948524 +0000 UTC m=+2275.483470620"
Jul 26 18:50:04.174324 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:04.174276    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl" podStartSLOduration=1.17425869 podStartE2EDuration="1.17425869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:04.173698299 +0000 UTC m=+2275.498220397" watchObservedRunningTime="2026-07-26 18:50:04.17425869 +0000 UTC m=+2275.498780786"
Jul 26 18:50:11.249069 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.249034    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"]
Jul 26 18:50:11.254155 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.254129    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:11.257125 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.257097    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-b9a04e\""
Jul 26 18:50:11.257125 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.257097    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-b9a04e-dockercfg-h2bjp\""
Jul 26 18:50:11.265262 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.265236    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"]
Jul 26 18:50:11.289470 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.289435    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be35ef6-5277-49d8-9490-ccb931c1d5bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg\" (UID: \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:11.289643 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.289500    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5fsd\" (UniqueName: \"kubernetes.io/projected/0be35ef6-5277-49d8-9490-ccb931c1d5bf-kube-api-access-q5fsd\") pod \"payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg\" (UID: \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:11.391032 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.390997    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be35ef6-5277-49d8-9490-ccb931c1d5bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg\" (UID: \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:11.391210 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.391052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5fsd\" (UniqueName: \"kubernetes.io/projected/0be35ef6-5277-49d8-9490-ccb931c1d5bf-kube-api-access-q5fsd\") pod \"payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg\" (UID: \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:11.391966 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.391938    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be35ef6-5277-49d8-9490-ccb931c1d5bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg\" (UID: \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:11.403841 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.403806    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5fsd\" (UniqueName: \"kubernetes.io/projected/0be35ef6-5277-49d8-9490-ccb931c1d5bf-kube-api-access-q5fsd\") pod \"payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg\" (UID: \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:11.566868 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.566765    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:11.914647 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:11.914606    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"]
Jul 26 18:50:11.916910 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:50:11.916885    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be35ef6_5277_49d8_9490_ccb931c1d5bf.slice/crio-e897239f9642f416513db96a3f02986e83c4d03a99760dbf2b91018923944d51 WatchSource:0}: Error finding container e897239f9642f416513db96a3f02986e83c4d03a99760dbf2b91018923944d51: Status 404 returned error can't find the container with id e897239f9642f416513db96a3f02986e83c4d03a99760dbf2b91018923944d51
Jul 26 18:50:12.184120 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:12.184031    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg" event={"ID":"0be35ef6-5277-49d8-9490-ccb931c1d5bf","Type":"ContainerStarted","Data":"503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322"}
Jul 26 18:50:12.184120 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:12.184091    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg" event={"ID":"0be35ef6-5277-49d8-9490-ccb931c1d5bf","Type":"ContainerStarted","Data":"e897239f9642f416513db96a3f02986e83c4d03a99760dbf2b91018923944d51"}
Jul 26 18:50:12.184360 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:12.184219    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:12.202438 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:12.202392    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg" podStartSLOduration=1.202378387 podStartE2EDuration="1.202378387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:12.20134626 +0000 UTC m=+2283.525868357" watchObservedRunningTime="2026-07-26 18:50:12.202378387 +0000 UTC m=+2283.526900537"
Jul 26 18:50:15.150115 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:15.150081    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:15.150595 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:15.150216    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:22.629179 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:22.629150    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"]
Jul 26 18:50:22.632208 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:22.632190    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"
Jul 26 18:50:22.634938 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:22.634916    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:50:22.635036 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:22.634916    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 18:50:22.639535 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:22.639512    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"]
Jul 26 18:50:22.700222 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:22.700195    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4759734-224d-41d2-b726-a149aef12cf1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-991a15-46d5k\" (UID: \"e4759734-224d-41d2-b726-a149aef12cf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"
Jul 26 18:50:22.800769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:22.800739    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4759734-224d-41d2-b726-a149aef12cf1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-991a15-46d5k\" (UID: \"e4759734-224d-41d2-b726-a149aef12cf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"
Jul 26 18:50:22.801381 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:22.801358    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4759734-224d-41d2-b726-a149aef12cf1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-991a15-46d5k\" (UID: \"e4759734-224d-41d2-b726-a149aef12cf1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"
Jul 26 18:50:22.944977 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:22.944944    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"
Jul 26 18:50:23.077495 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:23.077471    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"]
Jul 26 18:50:23.080196 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:50:23.080159    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4759734_224d_41d2_b726_a149aef12cf1.slice/crio-5ea23f063e40c1d1a89c05d16a6bdfc01ef1ab6d810c304b00d3724821a83c07 WatchSource:0}: Error finding container 5ea23f063e40c1d1a89c05d16a6bdfc01ef1ab6d810c304b00d3724821a83c07: Status 404 returned error can't find the container with id 5ea23f063e40c1d1a89c05d16a6bdfc01ef1ab6d810c304b00d3724821a83c07
Jul 26 18:50:23.190710 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:23.190688    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:50:23.240040 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:23.239838    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k" event={"ID":"e4759734-224d-41d2-b726-a149aef12cf1","Type":"ContainerStarted","Data":"5ea23f063e40c1d1a89c05d16a6bdfc01ef1ab6d810c304b00d3724821a83c07"}
Jul 26 18:50:24.248049 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:24.247962    2559 generic.go:356] "Generic (PLEG): container finished" podID="e4759734-224d-41d2-b726-a149aef12cf1" containerID="080e74df98327e3c32b91897b8fc46b5e5f06cd2f2804469f39212e099b457b8" exitCode=0
Jul 26 18:50:24.248413 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:24.248047    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k" event={"ID":"e4759734-224d-41d2-b726-a149aef12cf1","Type":"ContainerDied","Data":"080e74df98327e3c32b91897b8fc46b5e5f06cd2f2804469f39212e099b457b8"}
Jul 26 18:50:25.389916 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:25.389892    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"
Jul 26 18:50:25.526351 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:25.526265    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4759734-224d-41d2-b726-a149aef12cf1-maas-api-service-ca\") pod \"e4759734-224d-41d2-b726-a149aef12cf1\" (UID: \"e4759734-224d-41d2-b726-a149aef12cf1\") "
Jul 26 18:50:25.526591 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:25.526569    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4759734-224d-41d2-b726-a149aef12cf1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e4759734-224d-41d2-b726-a149aef12cf1" (UID: "e4759734-224d-41d2-b726-a149aef12cf1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:25.627556 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:25.627522    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4759734-224d-41d2-b726-a149aef12cf1-maas-api-service-ca\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:50:26.258130 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:26.258102    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"
Jul 26 18:50:26.258300 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:26.258097    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k" event={"ID":"e4759734-224d-41d2-b726-a149aef12cf1","Type":"ContainerDied","Data":"5ea23f063e40c1d1a89c05d16a6bdfc01ef1ab6d810c304b00d3724821a83c07"}
Jul 26 18:50:26.258300 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:26.258222    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ea23f063e40c1d1a89c05d16a6bdfc01ef1ab6d810c304b00d3724821a83c07"
Jul 26 18:50:37.979015 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:37.978874    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"]
Jul 26 18:50:37.979483 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:37.979136    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9" podUID="c23142c4-6fea-4b66-aee6-956629b7c268" containerName="payload-processing" containerID="cri-o://b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f" gracePeriod=30
Jul 26 18:50:37.986236 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:37.986146    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"]
Jul 26 18:50:37.986479 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:37.986410    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl" podUID="24b401e9-c07c-40d0-930e-dfb4a854032c" containerName="payload-pre-processing" containerID="cri-o://d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7" gracePeriod=30
Jul 26 18:50:38.249484 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.249459    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:38.269865 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.269843    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:38.316842 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.316809    2559 generic.go:356] "Generic (PLEG): container finished" podID="24b401e9-c07c-40d0-930e-dfb4a854032c" containerID="d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7" exitCode=0
Jul 26 18:50:38.316981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.316853    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"
Jul 26 18:50:38.316981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.316887    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl" event={"ID":"24b401e9-c07c-40d0-930e-dfb4a854032c","Type":"ContainerDied","Data":"d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7"}
Jul 26 18:50:38.316981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.316976    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl" event={"ID":"24b401e9-c07c-40d0-930e-dfb4a854032c","Type":"ContainerDied","Data":"9bf3d3a446ca718e10cd31814c3aa4dd10c94fa8e9d8f1d263413d8b9c2c62a2"}
Jul 26 18:50:38.317110 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.317007    2559 scope.go:117] "RemoveContainer" containerID="d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7"
Jul 26 18:50:38.318181 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.318160    2559 generic.go:356] "Generic (PLEG): container finished" podID="c23142c4-6fea-4b66-aee6-956629b7c268" containerID="b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f" exitCode=0
Jul 26 18:50:38.318296 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.318208    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"
Jul 26 18:50:38.318296 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.318228    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9" event={"ID":"c23142c4-6fea-4b66-aee6-956629b7c268","Type":"ContainerDied","Data":"b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f"}
Jul 26 18:50:38.318296 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.318266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9" event={"ID":"c23142c4-6fea-4b66-aee6-956629b7c268","Type":"ContainerDied","Data":"066233a6b784a1060cb467b269eb2e83a05b6b9ba1ca6f05e7206a6d2ccaa607"}
Jul 26 18:50:38.325599 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.325585    2559 scope.go:117] "RemoveContainer" containerID="d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7"
Jul 26 18:50:38.325914 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:50:38.325896    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7\": container with ID starting with d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7 not found: ID does not exist" containerID="d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7"
Jul 26 18:50:38.325972 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.325924    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7"} err="failed to get container status \"d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7\": rpc error: code = NotFound desc = could not find container \"d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7\": container with ID starting with d7191135354343b1c44e16131ea40ca1664b13a6b47a1b5dbf948222c78a47e7 not found: ID does not exist"
Jul 26 18:50:38.325972 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.325941    2559 scope.go:117] "RemoveContainer" containerID="b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f"
Jul 26 18:50:38.333738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.333710    2559 scope.go:117] "RemoveContainer" containerID="b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f"
Jul 26 18:50:38.333993 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:50:38.333973    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f\": container with ID starting with b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f not found: ID does not exist" containerID="b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f"
Jul 26 18:50:38.334061 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.334000    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f"} err="failed to get container status \"b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f\": rpc error: code = NotFound desc = could not find container \"b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f\": container with ID starting with b94d7911a00a5f50b88bd4db10ff4c1716d96178566f9ab2dcce97b639cac43f not found: ID does not exist"
Jul 26 18:50:38.356303 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.356281    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b401e9-c07c-40d0-930e-dfb4a854032c-plugins-config-volume\") pod \"24b401e9-c07c-40d0-930e-dfb4a854032c\" (UID: \"24b401e9-c07c-40d0-930e-dfb4a854032c\") "
Jul 26 18:50:38.356402 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.356325    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-824gd\" (UniqueName: \"kubernetes.io/projected/c23142c4-6fea-4b66-aee6-956629b7c268-kube-api-access-824gd\") pod \"c23142c4-6fea-4b66-aee6-956629b7c268\" (UID: \"c23142c4-6fea-4b66-aee6-956629b7c268\") "
Jul 26 18:50:38.356462 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.356433    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23142c4-6fea-4b66-aee6-956629b7c268-plugins-config-volume\") pod \"c23142c4-6fea-4b66-aee6-956629b7c268\" (UID: \"c23142c4-6fea-4b66-aee6-956629b7c268\") "
Jul 26 18:50:38.356462 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.356457    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fg8gk\" (UniqueName: \"kubernetes.io/projected/24b401e9-c07c-40d0-930e-dfb4a854032c-kube-api-access-fg8gk\") pod \"24b401e9-c07c-40d0-930e-dfb4a854032c\" (UID: \"24b401e9-c07c-40d0-930e-dfb4a854032c\") "
Jul 26 18:50:38.356756 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.356732    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24b401e9-c07c-40d0-930e-dfb4a854032c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "24b401e9-c07c-40d0-930e-dfb4a854032c" (UID: "24b401e9-c07c-40d0-930e-dfb4a854032c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:38.356860 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.356835    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23142c4-6fea-4b66-aee6-956629b7c268-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c23142c4-6fea-4b66-aee6-956629b7c268" (UID: "c23142c4-6fea-4b66-aee6-956629b7c268"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:38.358467 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.358445    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24b401e9-c07c-40d0-930e-dfb4a854032c-kube-api-access-fg8gk" (OuterVolumeSpecName: "kube-api-access-fg8gk") pod "24b401e9-c07c-40d0-930e-dfb4a854032c" (UID: "24b401e9-c07c-40d0-930e-dfb4a854032c"). InnerVolumeSpecName "kube-api-access-fg8gk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:38.358522 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.358481    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23142c4-6fea-4b66-aee6-956629b7c268-kube-api-access-824gd" (OuterVolumeSpecName: "kube-api-access-824gd") pod "c23142c4-6fea-4b66-aee6-956629b7c268" (UID: "c23142c4-6fea-4b66-aee6-956629b7c268"). InnerVolumeSpecName "kube-api-access-824gd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:50:38.457839 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.457812    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c23142c4-6fea-4b66-aee6-956629b7c268-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:50:38.457839 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.457836    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fg8gk\" (UniqueName: \"kubernetes.io/projected/24b401e9-c07c-40d0-930e-dfb4a854032c-kube-api-access-fg8gk\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:50:38.457981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.457847    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24b401e9-c07c-40d0-930e-dfb4a854032c-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:50:38.457981 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.457856    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-824gd\" (UniqueName: \"kubernetes.io/projected/c23142c4-6fea-4b66-aee6-956629b7c268-kube-api-access-824gd\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:50:38.646477 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.646443    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"]
Jul 26 18:50:38.650358 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.650332    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-991a15-74dcd8d556-vc5gl"]
Jul 26 18:50:38.661683 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.661656    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"]
Jul 26 18:50:38.663332 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:38.663310    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-991a15-8c9b4b5f6-9lsl9"]
Jul 26 18:50:39.176374 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:39.176342    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24b401e9-c07c-40d0-930e-dfb4a854032c" path="/var/lib/kubelet/pods/24b401e9-c07c-40d0-930e-dfb4a854032c/volumes"
Jul 26 18:50:39.176748 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:39.176713    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c23142c4-6fea-4b66-aee6-956629b7c268" path="/var/lib/kubelet/pods/c23142c4-6fea-4b66-aee6-956629b7c268/volumes"
Jul 26 18:50:42.713074 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:42.713033    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"]
Jul 26 18:50:42.715507 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:42.715482    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-991a15-46d5k"]
Jul 26 18:50:43.176313 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:43.176277    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4759734-224d-41d2-b726-a149aef12cf1" path="/var/lib/kubelet/pods/e4759734-224d-41d2-b726-a149aef12cf1/volumes"
Jul 26 18:50:49.327704 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.327666    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"]
Jul 26 18:50:49.328088 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.328051    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4759734-224d-41d2-b726-a149aef12cf1" containerName="revoke-keys"
Jul 26 18:50:49.328088 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.328062    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4759734-224d-41d2-b726-a149aef12cf1" containerName="revoke-keys"
Jul 26 18:50:49.328088 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.328078    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24b401e9-c07c-40d0-930e-dfb4a854032c" containerName="payload-pre-processing"
Jul 26 18:50:49.328088 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.328083    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="24b401e9-c07c-40d0-930e-dfb4a854032c" containerName="payload-pre-processing"
Jul 26 18:50:49.328229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.328095    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c23142c4-6fea-4b66-aee6-956629b7c268" containerName="payload-processing"
Jul 26 18:50:49.328229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.328100    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c23142c4-6fea-4b66-aee6-956629b7c268" containerName="payload-processing"
Jul 26 18:50:49.328229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.328179    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="24b401e9-c07c-40d0-930e-dfb4a854032c" containerName="payload-pre-processing"
Jul 26 18:50:49.328229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.328191    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c23142c4-6fea-4b66-aee6-956629b7c268" containerName="payload-processing"
Jul 26 18:50:49.328229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.328196    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4759734-224d-41d2-b726-a149aef12cf1" containerName="revoke-keys"
Jul 26 18:50:49.331026 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.330999    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"
Jul 26 18:50:49.333789 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.333748    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 18:50:49.333944 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.333816    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 18:50:49.340000 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.339565    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"]
Jul 26 18:50:49.361539 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.361490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c01669-8409-4405-a171-b77447e7ea6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr\" (UID: \"75c01669-8409-4405-a171-b77447e7ea6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"
Jul 26 18:50:49.462994 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.462955    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c01669-8409-4405-a171-b77447e7ea6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr\" (UID: \"75c01669-8409-4405-a171-b77447e7ea6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"
Jul 26 18:50:49.463546 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.463526    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c01669-8409-4405-a171-b77447e7ea6f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr\" (UID: \"75c01669-8409-4405-a171-b77447e7ea6f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"
Jul 26 18:50:49.645565 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.645462    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"
Jul 26 18:50:49.780080 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:49.780055    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"]
Jul 26 18:50:49.782530 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:50:49.782502    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c01669_8409_4405_a171_b77447e7ea6f.slice/crio-ce02dffda39f0b2d1449c6d0f338cf9c7a6cf294531c475f44144f63cf8776b4 WatchSource:0}: Error finding container ce02dffda39f0b2d1449c6d0f338cf9c7a6cf294531c475f44144f63cf8776b4: Status 404 returned error can't find the container with id ce02dffda39f0b2d1449c6d0f338cf9c7a6cf294531c475f44144f63cf8776b4
Jul 26 18:50:50.385143 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:50.385023    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr" event={"ID":"75c01669-8409-4405-a171-b77447e7ea6f","Type":"ContainerStarted","Data":"856079d102b7b2fee7e90ece24b1b410fa8bc0eff392b5c7623c0acab153a74d"}
Jul 26 18:50:50.385143 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:50.385092    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr" event={"ID":"75c01669-8409-4405-a171-b77447e7ea6f","Type":"ContainerStarted","Data":"ce02dffda39f0b2d1449c6d0f338cf9c7a6cf294531c475f44144f63cf8776b4"}
Jul 26 18:50:50.402028 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:50.401977    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr" podStartSLOduration=1.401960364 podStartE2EDuration="1.401960364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:50:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:50:50.400288772 +0000 UTC m=+2321.724810860" watchObservedRunningTime="2026-07-26 18:50:50.401960364 +0000 UTC m=+2321.726482454"
Jul 26 18:50:51.391953 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:51.391917    2559 generic.go:356] "Generic (PLEG): container finished" podID="75c01669-8409-4405-a171-b77447e7ea6f" containerID="856079d102b7b2fee7e90ece24b1b410fa8bc0eff392b5c7623c0acab153a74d" exitCode=0
Jul 26 18:50:51.392324 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:51.391973    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr" event={"ID":"75c01669-8409-4405-a171-b77447e7ea6f","Type":"ContainerDied","Data":"856079d102b7b2fee7e90ece24b1b410fa8bc0eff392b5c7623c0acab153a74d"}
Jul 26 18:50:52.525545 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:52.525522    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"
Jul 26 18:50:52.593138 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:52.593106    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c01669-8409-4405-a171-b77447e7ea6f-maas-api-service-ca\") pod \"75c01669-8409-4405-a171-b77447e7ea6f\" (UID: \"75c01669-8409-4405-a171-b77447e7ea6f\") "
Jul 26 18:50:52.593434 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:52.593411    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75c01669-8409-4405-a171-b77447e7ea6f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "75c01669-8409-4405-a171-b77447e7ea6f" (UID: "75c01669-8409-4405-a171-b77447e7ea6f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:50:52.694252 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:52.694219    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c01669-8409-4405-a171-b77447e7ea6f-maas-api-service-ca\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:50:53.402966 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:53.402938    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"
Jul 26 18:50:53.403119 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:53.402962    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr" event={"ID":"75c01669-8409-4405-a171-b77447e7ea6f","Type":"ContainerDied","Data":"ce02dffda39f0b2d1449c6d0f338cf9c7a6cf294531c475f44144f63cf8776b4"}
Jul 26 18:50:53.403119 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:50:53.403008    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce02dffda39f0b2d1449c6d0f338cf9c7a6cf294531c475f44144f63cf8776b4"
Jul 26 18:51:04.895065 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:04.894977    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"]
Jul 26 18:51:04.895575 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:04.895241    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg" podUID="0be35ef6-5277-49d8-9490-ccb931c1d5bf" containerName="payload-pre-processing" containerID="cri-o://503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322" gracePeriod=30
Jul 26 18:51:05.147253 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.147192    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:51:05.211276 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.211241    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be35ef6-5277-49d8-9490-ccb931c1d5bf-plugins-config-volume\") pod \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\" (UID: \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\") "
Jul 26 18:51:05.211425 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.211328    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5fsd\" (UniqueName: \"kubernetes.io/projected/0be35ef6-5277-49d8-9490-ccb931c1d5bf-kube-api-access-q5fsd\") pod \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\" (UID: \"0be35ef6-5277-49d8-9490-ccb931c1d5bf\") "
Jul 26 18:51:05.211598 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.211575    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be35ef6-5277-49d8-9490-ccb931c1d5bf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0be35ef6-5277-49d8-9490-ccb931c1d5bf" (UID: "0be35ef6-5277-49d8-9490-ccb931c1d5bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 18:51:05.213301 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.213281    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be35ef6-5277-49d8-9490-ccb931c1d5bf-kube-api-access-q5fsd" (OuterVolumeSpecName: "kube-api-access-q5fsd") pod "0be35ef6-5277-49d8-9490-ccb931c1d5bf" (UID: "0be35ef6-5277-49d8-9490-ccb931c1d5bf"). InnerVolumeSpecName "kube-api-access-q5fsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:51:05.312804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.312773    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0be35ef6-5277-49d8-9490-ccb931c1d5bf-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.312804 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.312802    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5fsd\" (UniqueName: \"kubernetes.io/projected/0be35ef6-5277-49d8-9490-ccb931c1d5bf-kube-api-access-q5fsd\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:51:05.464254 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.464224    2559 generic.go:356] "Generic (PLEG): container finished" podID="0be35ef6-5277-49d8-9490-ccb931c1d5bf" containerID="503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322" exitCode=0
Jul 26 18:51:05.464403 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.464267    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"
Jul 26 18:51:05.464403 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.464288    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg" event={"ID":"0be35ef6-5277-49d8-9490-ccb931c1d5bf","Type":"ContainerDied","Data":"503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322"}
Jul 26 18:51:05.464403 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.464332    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg" event={"ID":"0be35ef6-5277-49d8-9490-ccb931c1d5bf","Type":"ContainerDied","Data":"e897239f9642f416513db96a3f02986e83c4d03a99760dbf2b91018923944d51"}
Jul 26 18:51:05.464403 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.464354    2559 scope.go:117] "RemoveContainer" containerID="503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322"
Jul 26 18:51:05.473235 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.473216    2559 scope.go:117] "RemoveContainer" containerID="503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322"
Jul 26 18:51:05.473490 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:51:05.473472    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322\": container with ID starting with 503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322 not found: ID does not exist" containerID="503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322"
Jul 26 18:51:05.473541 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.473498    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322"} err="failed to get container status \"503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322\": rpc error: code = NotFound desc = could not find container \"503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322\": container with ID starting with 503022941c01b5eb1bda6261468f6b6a8ffed6c896afa18bf88ce3de908d0322 not found: ID does not exist"
Jul 26 18:51:05.486584 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.486529    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"]
Jul 26 18:51:05.489551 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:05.489530    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-b9a04e-6b854c6db4-h7mlg"]
Jul 26 18:51:07.176583 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:07.176551    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0be35ef6-5277-49d8-9490-ccb931c1d5bf" path="/var/lib/kubelet/pods/0be35ef6-5277-49d8-9490-ccb931c1d5bf/volumes"
Jul 26 18:51:09.513503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:09.513460    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"]
Jul 26 18:51:09.515629 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:09.515586    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-b9a04e-8prdr"]
Jul 26 18:51:11.176403 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:11.176369    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75c01669-8409-4405-a171-b77447e7ea6f" path="/var/lib/kubelet/pods/75c01669-8409-4405-a171-b77447e7ea6f/volumes"
Jul 26 18:51:20.500491 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.500453    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"]
Jul 26 18:51:20.500931 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.500884    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0be35ef6-5277-49d8-9490-ccb931c1d5bf" containerName="payload-pre-processing"
Jul 26 18:51:20.500931 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.500897    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0be35ef6-5277-49d8-9490-ccb931c1d5bf" containerName="payload-pre-processing"
Jul 26 18:51:20.500931 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.500930    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75c01669-8409-4405-a171-b77447e7ea6f" containerName="revoke-keys"
Jul 26 18:51:20.501052 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.500938    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="75c01669-8409-4405-a171-b77447e7ea6f" containerName="revoke-keys"
Jul 26 18:51:20.501052 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.500996    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0be35ef6-5277-49d8-9490-ccb931c1d5bf" containerName="payload-pre-processing"
Jul 26 18:51:20.501052 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.501007    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="75c01669-8409-4405-a171-b77447e7ea6f" containerName="revoke-keys"
Jul 26 18:51:20.503895 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.503879    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:20.506580 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.506560    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-2b193e\""
Jul 26 18:51:20.513063 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.513043    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"]
Jul 26 18:51:20.539755 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.539718    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"]
Jul 26 18:51:20.543784 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.543763    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:20.546761 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.546740    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-2b193e-dockercfg-klswx\""
Jul 26 18:51:20.552561 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.552529    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7hgd\" (UniqueName: \"kubernetes.io/projected/622ef703-a7d2-4556-adb3-45e2783901a8-kube-api-access-l7hgd\") pod \"payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b\" (UID: \"622ef703-a7d2-4556-adb3-45e2783901a8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:20.552690 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.552667    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4pk4\" (UniqueName: \"kubernetes.io/projected/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-kube-api-access-c4pk4\") pod \"payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn\" (UID: \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:20.552769 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.552730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn\" (UID: \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:20.552832 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.552772    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/622ef703-a7d2-4556-adb3-45e2783901a8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b\" (UID: \"622ef703-a7d2-4556-adb3-45e2783901a8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:20.553680 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.553660    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"]
Jul 26 18:51:20.654093 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.654056    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/622ef703-a7d2-4556-adb3-45e2783901a8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b\" (UID: \"622ef703-a7d2-4556-adb3-45e2783901a8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:20.654300 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.654131    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7hgd\" (UniqueName: \"kubernetes.io/projected/622ef703-a7d2-4556-adb3-45e2783901a8-kube-api-access-l7hgd\") pod \"payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b\" (UID: \"622ef703-a7d2-4556-adb3-45e2783901a8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:20.654300 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.654198    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4pk4\" (UniqueName: \"kubernetes.io/projected/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-kube-api-access-c4pk4\") pod \"payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn\" (UID: \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:20.654300 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.654219    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn\" (UID: \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:20.654903 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.654879    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/622ef703-a7d2-4556-adb3-45e2783901a8-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b\" (UID: \"622ef703-a7d2-4556-adb3-45e2783901a8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:20.655001 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.654880    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn\" (UID: \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:20.663692 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.663667    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4pk4\" (UniqueName: \"kubernetes.io/projected/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-kube-api-access-c4pk4\") pod \"payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn\" (UID: \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:20.663782 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.663743    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7hgd\" (UniqueName: \"kubernetes.io/projected/622ef703-a7d2-4556-adb3-45e2783901a8-kube-api-access-l7hgd\") pod \"payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b\" (UID: \"622ef703-a7d2-4556-adb3-45e2783901a8\") " pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:20.815095 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.815009    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:20.856471 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.856436    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:20.966645 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:20.966602    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"]
Jul 26 18:51:21.011792 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:21.011769    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"]
Jul 26 18:51:21.013706 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:51:21.013679    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622ef703_a7d2_4556_adb3_45e2783901a8.slice/crio-cbf5e4ec4429d75da9dd551fa02c16031a7bfccc84e2ca58a7d05fecba0559a8 WatchSource:0}: Error finding container cbf5e4ec4429d75da9dd551fa02c16031a7bfccc84e2ca58a7d05fecba0559a8: Status 404 returned error can't find the container with id cbf5e4ec4429d75da9dd551fa02c16031a7bfccc84e2ca58a7d05fecba0559a8
Jul 26 18:51:21.544915 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:21.544868    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn" event={"ID":"e12efcc8-f2ed-4ac3-aa3f-7764675d315c","Type":"ContainerStarted","Data":"41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476"}
Jul 26 18:51:21.545400 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:21.544935    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn" event={"ID":"e12efcc8-f2ed-4ac3-aa3f-7764675d315c","Type":"ContainerStarted","Data":"f2d14b61ca1ce9a77c4792c9840e8aad8f46aa47b04700b05adbcd21f6e11b69"}
Jul 26 18:51:21.545400 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:21.545058    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:21.546475 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:21.546448    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b" event={"ID":"622ef703-a7d2-4556-adb3-45e2783901a8","Type":"ContainerStarted","Data":"48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00"}
Jul 26 18:51:21.546607 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:21.546496    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b" event={"ID":"622ef703-a7d2-4556-adb3-45e2783901a8","Type":"ContainerStarted","Data":"cbf5e4ec4429d75da9dd551fa02c16031a7bfccc84e2ca58a7d05fecba0559a8"}
Jul 26 18:51:21.546607 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:21.546540    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:21.562412 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:21.562363    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn" podStartSLOduration=1.562347251 podStartE2EDuration="1.562347251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:21.560603053 +0000 UTC m=+2352.885125152" watchObservedRunningTime="2026-07-26 18:51:21.562347251 +0000 UTC m=+2352.886869347"
Jul 26 18:51:21.576219 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:21.576171    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b" podStartSLOduration=1.576157668 podStartE2EDuration="1.576157668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:21.575110805 +0000 UTC m=+2352.899632903" watchObservedRunningTime="2026-07-26 18:51:21.576157668 +0000 UTC m=+2352.900679764"
Jul 26 18:51:28.505450 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.505416    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"]
Jul 26 18:51:28.509221 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.509205    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:28.512483 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.512461    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-c75d5a\""
Jul 26 18:51:28.512557 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.512535    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-c75d5a-dockercfg-kws2f\""
Jul 26 18:51:28.521542 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.521513    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"]
Jul 26 18:51:28.547102 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.547073    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"]
Jul 26 18:51:28.550471 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.550452    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:28.559198 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.559175    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"]
Jul 26 18:51:28.630716 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.630673    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-784vc\" (UniqueName: \"kubernetes.io/projected/43fd86c7-c336-45f5-8952-a277646997fc-kube-api-access-784vc\") pod \"payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw\" (UID: \"43fd86c7-c336-45f5-8952-a277646997fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:28.630895 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.630764    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwqsz\" (UniqueName: \"kubernetes.io/projected/f86e051f-2f89-47d3-8880-acad32a4d2a3-kube-api-access-rwqsz\") pod \"payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh\" (UID: \"f86e051f-2f89-47d3-8880-acad32a4d2a3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:28.630895 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.630808    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43fd86c7-c336-45f5-8952-a277646997fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw\" (UID: \"43fd86c7-c336-45f5-8952-a277646997fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:28.630895 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.630859    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86e051f-2f89-47d3-8880-acad32a4d2a3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh\" (UID: \"f86e051f-2f89-47d3-8880-acad32a4d2a3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:28.732129 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.732093    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43fd86c7-c336-45f5-8952-a277646997fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw\" (UID: \"43fd86c7-c336-45f5-8952-a277646997fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:28.732316 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.732149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86e051f-2f89-47d3-8880-acad32a4d2a3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh\" (UID: \"f86e051f-2f89-47d3-8880-acad32a4d2a3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:28.732316 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.732226    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-784vc\" (UniqueName: \"kubernetes.io/projected/43fd86c7-c336-45f5-8952-a277646997fc-kube-api-access-784vc\") pod \"payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw\" (UID: \"43fd86c7-c336-45f5-8952-a277646997fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:28.732316 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.732293    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwqsz\" (UniqueName: \"kubernetes.io/projected/f86e051f-2f89-47d3-8880-acad32a4d2a3-kube-api-access-rwqsz\") pod \"payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh\" (UID: \"f86e051f-2f89-47d3-8880-acad32a4d2a3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:28.732973 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.732951    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43fd86c7-c336-45f5-8952-a277646997fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw\" (UID: \"43fd86c7-c336-45f5-8952-a277646997fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:28.732973 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.732961    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86e051f-2f89-47d3-8880-acad32a4d2a3-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh\" (UID: \"f86e051f-2f89-47d3-8880-acad32a4d2a3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:28.742080 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.742060    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-784vc\" (UniqueName: \"kubernetes.io/projected/43fd86c7-c336-45f5-8952-a277646997fc-kube-api-access-784vc\") pod \"payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw\" (UID: \"43fd86c7-c336-45f5-8952-a277646997fc\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:28.742186 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.742102    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwqsz\" (UniqueName: \"kubernetes.io/projected/f86e051f-2f89-47d3-8880-acad32a4d2a3-kube-api-access-rwqsz\") pod \"payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh\" (UID: \"f86e051f-2f89-47d3-8880-acad32a4d2a3\") " pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:28.821147 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.821072    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:28.861579 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:28.861547    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:29.002218 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.002185    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"]
Jul 26 18:51:29.003207 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:51:29.003178    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43fd86c7_c336_45f5_8952_a277646997fc.slice/crio-23878021b64effd1974028d26e61ea1f3618c009ff84aa3ebd4bd448c803eca7 WatchSource:0}: Error finding container 23878021b64effd1974028d26e61ea1f3618c009ff84aa3ebd4bd448c803eca7: Status 404 returned error can't find the container with id 23878021b64effd1974028d26e61ea1f3618c009ff84aa3ebd4bd448c803eca7
Jul 26 18:51:29.038482 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.038454    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"]
Jul 26 18:51:29.041164 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:51:29.041142    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf86e051f_2f89_47d3_8880_acad32a4d2a3.slice/crio-bcfa23429943850e6fa0b1e887d637dc3a43fba3598dcdf307f5180a2514360c WatchSource:0}: Error finding container bcfa23429943850e6fa0b1e887d637dc3a43fba3598dcdf307f5180a2514360c: Status 404 returned error can't find the container with id bcfa23429943850e6fa0b1e887d637dc3a43fba3598dcdf307f5180a2514360c
Jul 26 18:51:29.589974 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.589942    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw" event={"ID":"43fd86c7-c336-45f5-8952-a277646997fc","Type":"ContainerStarted","Data":"a8bee1a8afda8d98d776eb8103794024afe9b6cc260511b4efe4b44522e82bbf"}
Jul 26 18:51:29.590485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.590007    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:29.590485 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.590029    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw" event={"ID":"43fd86c7-c336-45f5-8952-a277646997fc","Type":"ContainerStarted","Data":"23878021b64effd1974028d26e61ea1f3618c009ff84aa3ebd4bd448c803eca7"}
Jul 26 18:51:29.591539 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.591515    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh" event={"ID":"f86e051f-2f89-47d3-8880-acad32a4d2a3","Type":"ContainerStarted","Data":"e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f"}
Jul 26 18:51:29.591669 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.591556    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh" event={"ID":"f86e051f-2f89-47d3-8880-acad32a4d2a3","Type":"ContainerStarted","Data":"bcfa23429943850e6fa0b1e887d637dc3a43fba3598dcdf307f5180a2514360c"}
Jul 26 18:51:29.591669 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.591585    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:29.606075 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.606013    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw" podStartSLOduration=1.605998503 podStartE2EDuration="1.605998503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:29.605184944 +0000 UTC m=+2360.929707046" watchObservedRunningTime="2026-07-26 18:51:29.605998503 +0000 UTC m=+2360.930520603"
Jul 26 18:51:29.622846 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:29.622796    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh" podStartSLOduration=1.622782417 podStartE2EDuration="1.622782417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:29.620905104 +0000 UTC m=+2360.945427202" watchObservedRunningTime="2026-07-26 18:51:29.622782417 +0000 UTC m=+2360.947304513"
Jul 26 18:51:32.555178 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:32.555146    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 18:51:32.555587 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:32.555213    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 18:51:40.597796 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:40.597761    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 18:51:40.598229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:40.597831    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 18:51:51.294877 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.294844    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"]
Jul 26 18:51:51.298341 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.298319    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.301336 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.301306    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"kube-root-ca.crt\""
Jul 26 18:51:51.301336 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.301314    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"auth-test-model-2b193e-kserve-self-signed-certs\""
Jul 26 18:51:51.302723 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.302704    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"openshift-service-ca.crt\""
Jul 26 18:51:51.302874 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.302770    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"default-dockercfg-cpbb9\""
Jul 26 18:51:51.308781 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.308757    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"]
Jul 26 18:51:51.455115 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.455080    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-home\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.455276 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.455129    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a91a418f-382e-4158-84de-56373cda103c-tls-certs\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.455276 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.455213    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-model-cache\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.455397 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.455304    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqxml\" (UniqueName: \"kubernetes.io/projected/a91a418f-382e-4158-84de-56373cda103c-kube-api-access-jqxml\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.455397 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.455348    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-tmp-dir\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.455502 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.455398    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-dshm\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.556935 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.556849    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqxml\" (UniqueName: \"kubernetes.io/projected/a91a418f-382e-4158-84de-56373cda103c-kube-api-access-jqxml\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.556935 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.556886    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-tmp-dir\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.556935 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.556913    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-dshm\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.557207 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.556947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-home\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.557207 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.556993    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a91a418f-382e-4158-84de-56373cda103c-tls-certs\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.557207 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.557067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-model-cache\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.557379 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.557319    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-tmp-dir\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.557429 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.557373    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-home\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.557465 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.557449    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-model-cache\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.559440 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.559416    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-dshm\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.559657 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.559639    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a91a418f-382e-4158-84de-56373cda103c-tls-certs\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.564782 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.564760    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqxml\" (UniqueName: \"kubernetes.io/projected/a91a418f-382e-4158-84de-56373cda103c-kube-api-access-jqxml\") pod \"auth-test-model-2b193e-kserve-84759ff787-ckllb\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") " pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.610503 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.610481    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:51:51.745307 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:51.745277    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"]
Jul 26 18:51:51.747303 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:51:51.747276    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda91a418f_382e_4158_84de_56373cda103c.slice/crio-c053a4020d98708278f321029404bf68e749e40bde6efb89e68f3b10a05dd385 WatchSource:0}: Error finding container c053a4020d98708278f321029404bf68e749e40bde6efb89e68f3b10a05dd385: Status 404 returned error can't find the container with id c053a4020d98708278f321029404bf68e749e40bde6efb89e68f3b10a05dd385
Jul 26 18:51:52.722422 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:52.722382    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb" event={"ID":"a91a418f-382e-4158-84de-56373cda103c","Type":"ContainerStarted","Data":"a966be5ba78544520d3d05b4ce6cc8af6f8e1aca992878571b50cf28fc7e2d55"}
Jul 26 18:51:52.722812 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:52.722431    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb" event={"ID":"a91a418f-382e-4158-84de-56373cda103c","Type":"ContainerStarted","Data":"c053a4020d98708278f321029404bf68e749e40bde6efb89e68f3b10a05dd385"}
Jul 26 18:51:52.742028 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:51:52.741979    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb" podStartSLOduration=1.741963819 podStartE2EDuration="1.741963819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:51:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:51:52.740431201 +0000 UTC m=+2384.064953310" watchObservedRunningTime="2026-07-26 18:51:52.741963819 +0000 UTC m=+2384.066485965"
Jul 26 18:52:01.611368 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:01.611317    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:52:01.611368 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:01.611378    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:52:01.624080 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:01.624052    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:52:01.781248 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:01.781222    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:52:09.951944 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:09.951815    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:52:09.967508 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:09.961427    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:52:14.100002 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.099968    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"]
Jul 26 18:52:14.103560 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.103539    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.107720 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.107697    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"default-dockercfg-w44r6\""
Jul 26 18:52:14.107869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.107753    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"kube-root-ca.crt\""
Jul 26 18:52:14.107869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.107760    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"auth-test-model-c75d5a-kserve-self-signed-certs\""
Jul 26 18:52:14.107988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.107972    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"openshift-service-ca.crt\""
Jul 26 18:52:14.112905 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.112885    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"]
Jul 26 18:52:14.158095 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.158071    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-dshm\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.158233 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.158105    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45285c81-be25-44bd-8968-2075c989ca10-tls-certs\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.158233 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.158181    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-home\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.158233 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.158225    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-model-cache\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.158369 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.158244    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-tmp-dir\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.158411 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.158386    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsklr\" (UniqueName: \"kubernetes.io/projected/45285c81-be25-44bd-8968-2075c989ca10-kube-api-access-wsklr\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.259050 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.259018    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-home\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.259188 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.259084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-model-cache\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.259188 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.259110    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-tmp-dir\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.259356 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.259211    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsklr\" (UniqueName: \"kubernetes.io/projected/45285c81-be25-44bd-8968-2075c989ca10-kube-api-access-wsklr\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.259356 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.259294    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-dshm\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.259356 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.259320    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45285c81-be25-44bd-8968-2075c989ca10-tls-certs\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.259518 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.259464    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-home\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.259583 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.259511    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-model-cache\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.259666 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.259587    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-tmp-dir\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.261524 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.261496    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-dshm\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.261754 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.261737    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45285c81-be25-44bd-8968-2075c989ca10-tls-certs\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.268116 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.268098    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsklr\" (UniqueName: \"kubernetes.io/projected/45285c81-be25-44bd-8968-2075c989ca10-kube-api-access-wsklr\") pod \"auth-test-model-c75d5a-kserve-558bf479d7-kvnr7\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") " pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.413896 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.413869    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:14.544394 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.544358    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"]
Jul 26 18:52:14.546382 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:52:14.546349    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45285c81_be25_44bd_8968_2075c989ca10.slice/crio-e27166d7d5b119d97f247aba4fc878ce7956b1f90c5d764b630ef2f754fd4d65 WatchSource:0}: Error finding container e27166d7d5b119d97f247aba4fc878ce7956b1f90c5d764b630ef2f754fd4d65: Status 404 returned error can't find the container with id e27166d7d5b119d97f247aba4fc878ce7956b1f90c5d764b630ef2f754fd4d65
Jul 26 18:52:14.846088 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.845984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7" event={"ID":"45285c81-be25-44bd-8968-2075c989ca10","Type":"ContainerStarted","Data":"437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64"}
Jul 26 18:52:14.846088 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.846060    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7" event={"ID":"45285c81-be25-44bd-8968-2075c989ca10","Type":"ContainerStarted","Data":"e27166d7d5b119d97f247aba4fc878ce7956b1f90c5d764b630ef2f754fd4d65"}
Jul 26 18:52:14.863592 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:14.863534    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7" podStartSLOduration=0.863520014 podStartE2EDuration="863.520014ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:52:14.861856957 +0000 UTC m=+2406.186379065" watchObservedRunningTime="2026-07-26 18:52:14.863520014 +0000 UTC m=+2406.188042110"
Jul 26 18:52:24.414734 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:24.414698    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:24.414734 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:24.414743    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:24.427670 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:24.427645    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:52:24.906867 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:52:24.906787    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:56:09.416125 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:09.416089    2559 scope.go:117] "RemoveContainer" containerID="cd19da57af09a32c71025bc9a5f80d8f1a0f5eccb549e3a37333633ac874a42a"
Jul 26 18:56:41.102328 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:41.102296    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"]
Jul 26 18:56:42.003869 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:42.003840    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"]
Jul 26 18:56:56.120701 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.120660    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb" podUID="a91a418f-382e-4158-84de-56373cda103c" containerName="main" containerID="cri-o://a966be5ba78544520d3d05b4ce6cc8af6f8e1aca992878571b50cf28fc7e2d55" gracePeriod=45
Jul 26 18:56:56.274151 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.274120    2559 generic.go:356] "Generic (PLEG): container finished" podID="a91a418f-382e-4158-84de-56373cda103c" containerID="a966be5ba78544520d3d05b4ce6cc8af6f8e1aca992878571b50cf28fc7e2d55" exitCode=0
Jul 26 18:56:56.274321 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.274206    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb" event={"ID":"a91a418f-382e-4158-84de-56373cda103c","Type":"ContainerDied","Data":"a966be5ba78544520d3d05b4ce6cc8af6f8e1aca992878571b50cf28fc7e2d55"}
Jul 26 18:56:56.365639 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.365604    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:56:56.406316 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406233    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-dshm\") pod \"a91a418f-382e-4158-84de-56373cda103c\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") "
Jul 26 18:56:56.406316 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406308    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-model-cache\") pod \"a91a418f-382e-4158-84de-56373cda103c\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") "
Jul 26 18:56:56.406518 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406345    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a91a418f-382e-4158-84de-56373cda103c-tls-certs\") pod \"a91a418f-382e-4158-84de-56373cda103c\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") "
Jul 26 18:56:56.406518 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406369    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-home\") pod \"a91a418f-382e-4158-84de-56373cda103c\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") "
Jul 26 18:56:56.406518 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406443    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-tmp-dir\") pod \"a91a418f-382e-4158-84de-56373cda103c\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") "
Jul 26 18:56:56.406518 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406500    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqxml\" (UniqueName: \"kubernetes.io/projected/a91a418f-382e-4158-84de-56373cda103c-kube-api-access-jqxml\") pod \"a91a418f-382e-4158-84de-56373cda103c\" (UID: \"a91a418f-382e-4158-84de-56373cda103c\") "
Jul 26 18:56:56.406742 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406532    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-model-cache" (OuterVolumeSpecName: "model-cache") pod "a91a418f-382e-4158-84de-56373cda103c" (UID: "a91a418f-382e-4158-84de-56373cda103c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:56:56.406742 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406714    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-home" (OuterVolumeSpecName: "home") pod "a91a418f-382e-4158-84de-56373cda103c" (UID: "a91a418f-382e-4158-84de-56373cda103c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:56:56.406917 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406865    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:56.406917 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406872    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a91a418f-382e-4158-84de-56373cda103c" (UID: "a91a418f-382e-4158-84de-56373cda103c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:56:56.406917 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.406891    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:56.408679 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.408648    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a91a418f-382e-4158-84de-56373cda103c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a91a418f-382e-4158-84de-56373cda103c" (UID: "a91a418f-382e-4158-84de-56373cda103c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:56:56.408806 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.408772    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a91a418f-382e-4158-84de-56373cda103c-kube-api-access-jqxml" (OuterVolumeSpecName: "kube-api-access-jqxml") pod "a91a418f-382e-4158-84de-56373cda103c" (UID: "a91a418f-382e-4158-84de-56373cda103c"). InnerVolumeSpecName "kube-api-access-jqxml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:56:56.408859 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.408815    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-dshm" (OuterVolumeSpecName: "dshm") pod "a91a418f-382e-4158-84de-56373cda103c" (UID: "a91a418f-382e-4158-84de-56373cda103c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:56:56.508229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.508202    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqxml\" (UniqueName: \"kubernetes.io/projected/a91a418f-382e-4158-84de-56373cda103c-kube-api-access-jqxml\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:56.508229 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.508226    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:56.508452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.508237    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a91a418f-382e-4158-84de-56373cda103c-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:56.508452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:56.508248    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a91a418f-382e-4158-84de-56373cda103c-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:57.023649 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.023571    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7" podUID="45285c81-be25-44bd-8968-2075c989ca10" containerName="main" containerID="cri-o://437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64" gracePeriod=45
Jul 26 18:56:57.279977 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.279922    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:56:57.280680 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.280663    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"
Jul 26 18:56:57.280753 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.280713    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb" event={"ID":"a91a418f-382e-4158-84de-56373cda103c","Type":"ContainerDied","Data":"c053a4020d98708278f321029404bf68e749e40bde6efb89e68f3b10a05dd385"}
Jul 26 18:56:57.280813 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.280762    2559 scope.go:117] "RemoveContainer" containerID="a966be5ba78544520d3d05b4ce6cc8af6f8e1aca992878571b50cf28fc7e2d55"
Jul 26 18:56:57.282059 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.282037    2559 generic.go:356] "Generic (PLEG): container finished" podID="45285c81-be25-44bd-8968-2075c989ca10" containerID="437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64" exitCode=0
Jul 26 18:56:57.282147 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.282070    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7" event={"ID":"45285c81-be25-44bd-8968-2075c989ca10","Type":"ContainerDied","Data":"437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64"}
Jul 26 18:56:57.282147 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.282083    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"
Jul 26 18:56:57.282147 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.282093    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7" event={"ID":"45285c81-be25-44bd-8968-2075c989ca10","Type":"ContainerDied","Data":"e27166d7d5b119d97f247aba4fc878ce7956b1f90c5d764b630ef2f754fd4d65"}
Jul 26 18:56:57.289312 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.289296    2559 scope.go:117] "RemoveContainer" containerID="437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64"
Jul 26 18:56:57.296579 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.296561    2559 scope.go:117] "RemoveContainer" containerID="437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64"
Jul 26 18:56:57.297682 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:56:57.296898    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64\": container with ID starting with 437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64 not found: ID does not exist" containerID="437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64"
Jul 26 18:56:57.297682 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.296937    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64"} err="failed to get container status \"437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64\": rpc error: code = NotFound desc = could not find container \"437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64\": container with ID starting with 437fffb0374562f4fbc527f5512a1a3a8a7e00ffac39932cda5aa03c610cfd64 not found: ID does not exist"
Jul 26 18:56:57.314321 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.314300    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"]
Jul 26 18:56:57.319834 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.319801    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/auth-test-model-2b193e-kserve-84759ff787-ckllb"]
Jul 26 18:56:57.417020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.416986    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-model-cache\") pod \"45285c81-be25-44bd-8968-2075c989ca10\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") "
Jul 26 18:56:57.417020 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417023    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsklr\" (UniqueName: \"kubernetes.io/projected/45285c81-be25-44bd-8968-2075c989ca10-kube-api-access-wsklr\") pod \"45285c81-be25-44bd-8968-2075c989ca10\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") "
Jul 26 18:56:57.417225 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417092    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45285c81-be25-44bd-8968-2075c989ca10-tls-certs\") pod \"45285c81-be25-44bd-8968-2075c989ca10\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") "
Jul 26 18:56:57.417225 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417118    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-tmp-dir\") pod \"45285c81-be25-44bd-8968-2075c989ca10\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") "
Jul 26 18:56:57.417225 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417148    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-home\") pod \"45285c81-be25-44bd-8968-2075c989ca10\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") "
Jul 26 18:56:57.417225 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417218    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-dshm\") pod \"45285c81-be25-44bd-8968-2075c989ca10\" (UID: \"45285c81-be25-44bd-8968-2075c989ca10\") "
Jul 26 18:56:57.417429 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417283    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-model-cache" (OuterVolumeSpecName: "model-cache") pod "45285c81-be25-44bd-8968-2075c989ca10" (UID: "45285c81-be25-44bd-8968-2075c989ca10"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:56:57.417429 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417347    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "45285c81-be25-44bd-8968-2075c989ca10" (UID: "45285c81-be25-44bd-8968-2075c989ca10"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:56:57.417531 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417428    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-home" (OuterVolumeSpecName: "home") pod "45285c81-be25-44bd-8968-2075c989ca10" (UID: "45285c81-be25-44bd-8968-2075c989ca10"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:56:57.417610 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417591    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:57.417679 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417641    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:57.417679 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.417656    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:57.419404 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.419382    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45285c81-be25-44bd-8968-2075c989ca10-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "45285c81-be25-44bd-8968-2075c989ca10" (UID: "45285c81-be25-44bd-8968-2075c989ca10"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:56:57.419404 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.419398    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-dshm" (OuterVolumeSpecName: "dshm") pod "45285c81-be25-44bd-8968-2075c989ca10" (UID: "45285c81-be25-44bd-8968-2075c989ca10"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:56:57.419535 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.419430    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45285c81-be25-44bd-8968-2075c989ca10-kube-api-access-wsklr" (OuterVolumeSpecName: "kube-api-access-wsklr") pod "45285c81-be25-44bd-8968-2075c989ca10" (UID: "45285c81-be25-44bd-8968-2075c989ca10"). InnerVolumeSpecName "kube-api-access-wsklr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:56:57.518971 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.518933    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45285c81-be25-44bd-8968-2075c989ca10-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:57.518971 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.518971    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsklr\" (UniqueName: \"kubernetes.io/projected/45285c81-be25-44bd-8968-2075c989ca10-kube-api-access-wsklr\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:57.518971 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.518982    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45285c81-be25-44bd-8968-2075c989ca10-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:56:57.603749 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.603718    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"]
Jul 26 18:56:57.607337 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:57.607313    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/auth-test-model-c75d5a-kserve-558bf479d7-kvnr7"]
Jul 26 18:56:59.176821 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:59.176789    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45285c81-be25-44bd-8968-2075c989ca10" path="/var/lib/kubelet/pods/45285c81-be25-44bd-8968-2075c989ca10/volumes"
Jul 26 18:56:59.177195 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:56:59.177176    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a91a418f-382e-4158-84de-56373cda103c" path="/var/lib/kubelet/pods/a91a418f-382e-4158-84de-56373cda103c/volumes"
Jul 26 18:57:00.716276 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.716193    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"]
Jul 26 18:57:00.716663 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.716587    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91a418f-382e-4158-84de-56373cda103c" containerName="main"
Jul 26 18:57:00.716663 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.716597    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91a418f-382e-4158-84de-56373cda103c" containerName="main"
Jul 26 18:57:00.716663 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.716633    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45285c81-be25-44bd-8968-2075c989ca10" containerName="main"
Jul 26 18:57:00.716663 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.716642    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="45285c81-be25-44bd-8968-2075c989ca10" containerName="main"
Jul 26 18:57:00.716810 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.716703    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91a418f-382e-4158-84de-56373cda103c" containerName="main"
Jul 26 18:57:00.716810 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.716713    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="45285c81-be25-44bd-8968-2075c989ca10" containerName="main"
Jul 26 18:57:00.721457 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.721441    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.724525 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.724506    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"kube-root-ca.crt\""
Jul 26 18:57:00.726064 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.725887    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"openshift-service-ca.crt\""
Jul 26 18:57:00.726064 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.725904    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"sub-test-model-c75d5a-kserve-self-signed-certs\""
Jul 26 18:57:00.726064 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.726044    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"default-dockercfg-w44r6\""
Jul 26 18:57:00.730169 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.730147    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"]
Jul 26 18:57:00.849685 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.849611    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd7st\" (UniqueName: \"kubernetes.io/projected/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-kube-api-access-hd7st\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.849833 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.849712    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-home\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.849833 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.849752    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-dshm\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.849833 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.849785    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tmp-dir\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.849833 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.849801    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tls-certs\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.849988 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.849909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-model-cache\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.951144 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.951109    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd7st\" (UniqueName: \"kubernetes.io/projected/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-kube-api-access-hd7st\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.951144 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.951144    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-home\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.951334 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.951168    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-dshm\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.951334 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.951187    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tmp-dir\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.951334 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.951207    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tls-certs\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.951334 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.951267    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-model-cache\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.951589 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.951562    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-home\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.951589 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.951582    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tmp-dir\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.951716 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.951670    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-model-cache\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.953377 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.953352    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-dshm\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.953547 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.953531    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tls-certs\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:00.959673 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:00.959650    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd7st\" (UniqueName: \"kubernetes.io/projected/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-kube-api-access-hd7st\") pod \"sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") " pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:01.032493 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:01.032429    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:01.163268 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:01.163243    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"]
Jul 26 18:57:01.165505 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:57:01.165476    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fab4fdb_4ea0_4f0d_a78e_04c1a77a2be0.slice/crio-d5252e92f510ec0ca4f8c32a7ec5040f11eca61864f850376240c89b244e953f WatchSource:0}: Error finding container d5252e92f510ec0ca4f8c32a7ec5040f11eca61864f850376240c89b244e953f: Status 404 returned error can't find the container with id d5252e92f510ec0ca4f8c32a7ec5040f11eca61864f850376240c89b244e953f
Jul 26 18:57:01.305337 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:01.305258    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t" event={"ID":"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0","Type":"ContainerStarted","Data":"bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08"}
Jul 26 18:57:01.305337 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:01.305296    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t" event={"ID":"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0","Type":"ContainerStarted","Data":"d5252e92f510ec0ca4f8c32a7ec5040f11eca61864f850376240c89b244e953f"}
Jul 26 18:57:01.323391 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:01.323339    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t" podStartSLOduration=1.323325091 podStartE2EDuration="1.323325091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:57:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:57:01.321068916 +0000 UTC m=+2692.645591025" watchObservedRunningTime="2026-07-26 18:57:01.323325091 +0000 UTC m=+2692.647847233"
Jul 26 18:57:09.435932 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:09.435901    2559 scope.go:117] "RemoveContainer" containerID="856079d102b7b2fee7e90ece24b1b410fa8bc0eff392b5c7623c0acab153a74d"
Jul 26 18:57:09.445203 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:09.445183    2559 scope.go:117] "RemoveContainer" containerID="080e74df98327e3c32b91897b8fc46b5e5f06cd2f2804469f39212e099b457b8"
Jul 26 18:57:09.982972 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:09.982857    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:57:10.000566 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:09.992862    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 18:57:11.033253 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:11.033217    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:11.033738 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:11.033269    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:11.045462 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:11.045441    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:57:11.372641 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:57:11.372542    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:58:19.183690 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:19.183660    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"]
Jul 26 18:58:34.202985 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.202876    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t" podUID="6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" containerName="main" containerID="cri-o://bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08" gracePeriod=45
Jul 26 18:58:34.461004 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.460942    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:58:34.558096 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558025    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tmp-dir\") pod \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") "
Jul 26 18:58:34.558096 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558062    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-model-cache\") pod \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") "
Jul 26 18:58:34.558350 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558111    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hd7st\" (UniqueName: \"kubernetes.io/projected/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-kube-api-access-hd7st\") pod \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") "
Jul 26 18:58:34.558350 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558136    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-dshm\") pod \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") "
Jul 26 18:58:34.558350 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558164    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-home\") pod \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") "
Jul 26 18:58:34.558350 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558205    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tls-certs\") pod \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\" (UID: \"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0\") "
Jul 26 18:58:34.558571 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558370    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" (UID: "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:58:34.558571 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558432    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-model-cache" (OuterVolumeSpecName: "model-cache") pod "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" (UID: "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:58:34.558571 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558518    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-home" (OuterVolumeSpecName: "home") pod "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" (UID: "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:58:34.558768 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558658    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:58:34.558768 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558676    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:58:34.558768 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.558689    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:58:34.560321 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.560294    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-dshm" (OuterVolumeSpecName: "dshm") pod "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" (UID: "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 18:58:34.560424 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.560341    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-kube-api-access-hd7st" (OuterVolumeSpecName: "kube-api-access-hd7st") pod "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" (UID: "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0"). InnerVolumeSpecName "kube-api-access-hd7st". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 18:58:34.560424 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.560360    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" (UID: "6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 18:58:34.660215 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.660191    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd7st\" (UniqueName: \"kubernetes.io/projected/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-kube-api-access-hd7st\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:58:34.660215 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.660214    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:58:34.660372 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.660228    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 18:58:34.789327 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.789239    2559 generic.go:356] "Generic (PLEG): container finished" podID="6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" containerID="bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08" exitCode=0
Jul 26 18:58:34.789327 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.789291    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"
Jul 26 18:58:34.789327 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.789316    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t" event={"ID":"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0","Type":"ContainerDied","Data":"bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08"}
Jul 26 18:58:34.789551 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.789353    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t" event={"ID":"6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0","Type":"ContainerDied","Data":"d5252e92f510ec0ca4f8c32a7ec5040f11eca61864f850376240c89b244e953f"}
Jul 26 18:58:34.789551 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.789374    2559 scope.go:117] "RemoveContainer" containerID="bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08"
Jul 26 18:58:34.804659 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.804603    2559 scope.go:117] "RemoveContainer" containerID="bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08"
Jul 26 18:58:34.804907 ip-10-0-138-70 kubenswrapper[2559]: E0726 18:58:34.804887    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08\": container with ID starting with bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08 not found: ID does not exist" containerID="bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08"
Jul 26 18:58:34.804967 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.804915    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08"} err="failed to get container status \"bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08\": rpc error: code = NotFound desc = could not find container \"bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08\": container with ID starting with bb1f73961f2eac2e66c8ef50593fb5feefcbde4824a2b45d242c484f1cd1cb08 not found: ID does not exist"
Jul 26 18:58:34.819482 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.819454    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"]
Jul 26 18:58:34.824393 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:34.824373    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/sub-test-model-c75d5a-kserve-65ddb5c6cc-jb99t"]
Jul 26 18:58:35.176610 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:35.176574    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" path="/var/lib/kubelet/pods/6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0/volumes"
Jul 26 18:58:41.598142 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.598109    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"]
Jul 26 18:58:41.598540 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.598500    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" containerName="main"
Jul 26 18:58:41.598540 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.598513    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" containerName="main"
Jul 26 18:58:41.598686 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.598669    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fab4fdb-4ea0-4f0d-a78e-04c1a77a2be0" containerName="main"
Jul 26 18:58:41.603537 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.603521    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.606562 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.606537    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"kube-root-ca.crt\""
Jul 26 18:58:41.608145 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.608126    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"default-dockercfg-cpbb9\""
Jul 26 18:58:41.608261 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.608124    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"rate-test-model-2b193e-kserve-self-signed-certs\""
Jul 26 18:58:41.608261 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.608175    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2b193e\"/\"openshift-service-ca.crt\""
Jul 26 18:58:41.612254 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.612233    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"]
Jul 26 18:58:41.721590 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.721562    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25kph\" (UniqueName: \"kubernetes.io/projected/60b36419-e171-402f-bb44-2e1234ce56d1-kube-api-access-25kph\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.721747 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.721598    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-model-cache\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.721747 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.721641    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b36419-e171-402f-bb44-2e1234ce56d1-tls-certs\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.721747 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.721691    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-home\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.721747 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.721735    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-tmp-dir\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.721889 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.721784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-dshm\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.823276 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.823242    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-home\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.823276 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.823277    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-tmp-dir\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.823536 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.823301    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-dshm\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.823536 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.823340    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25kph\" (UniqueName: \"kubernetes.io/projected/60b36419-e171-402f-bb44-2e1234ce56d1-kube-api-access-25kph\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.823536 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.823363    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-model-cache\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.823536 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.823383    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b36419-e171-402f-bb44-2e1234ce56d1-tls-certs\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.823786 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.823731    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-home\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.823786 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.823742    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-tmp-dir\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.823925 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.823907    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-model-cache\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.825734 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.825709    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-dshm\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.826313 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.826296    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b36419-e171-402f-bb44-2e1234ce56d1-tls-certs\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.831929 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.831910    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25kph\" (UniqueName: \"kubernetes.io/projected/60b36419-e171-402f-bb44-2e1234ce56d1-kube-api-access-25kph\") pod \"rate-test-model-2b193e-kserve-c8d57fff-zzjbz\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") " pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:41.914534 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:41.914509    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:42.046658 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:42.046628    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"]
Jul 26 18:58:42.047946 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:58:42.047917    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60b36419_e171_402f_bb44_2e1234ce56d1.slice/crio-8114daf96cad00e519fdbc7e2bf43d5f7e6713857d7aace6fe63a10b1052d3c3 WatchSource:0}: Error finding container 8114daf96cad00e519fdbc7e2bf43d5f7e6713857d7aace6fe63a10b1052d3c3: Status 404 returned error can't find the container with id 8114daf96cad00e519fdbc7e2bf43d5f7e6713857d7aace6fe63a10b1052d3c3
Jul 26 18:58:42.834243 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:42.834197    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz" event={"ID":"60b36419-e171-402f-bb44-2e1234ce56d1","Type":"ContainerStarted","Data":"cca03ea0284d14d6f6caa9e5444dd41cf36fab5d3b3f606c92d0ff539ca26443"}
Jul 26 18:58:42.834737 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:42.834250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz" event={"ID":"60b36419-e171-402f-bb44-2e1234ce56d1","Type":"ContainerStarted","Data":"8114daf96cad00e519fdbc7e2bf43d5f7e6713857d7aace6fe63a10b1052d3c3"}
Jul 26 18:58:42.852311 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:42.852265    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz" podStartSLOduration=1.85224933 podStartE2EDuration="1.85224933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:58:42.851581104 +0000 UTC m=+2794.176103202" watchObservedRunningTime="2026-07-26 18:58:42.85224933 +0000 UTC m=+2794.176771426"
Jul 26 18:58:51.914797 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:51.914758    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:51.915178 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:51.914810    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:51.927603 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:51.927581    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:58:52.895309 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:58:52.895281    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 18:59:00.097249 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.097218    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"]
Jul 26 18:59:00.100870 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.100853    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.105162 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.105120    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"rate-test-model-c75d5a-kserve-self-signed-certs\""
Jul 26 18:59:00.105344 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.105194    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"kube-root-ca.crt\""
Jul 26 18:59:00.105344 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.105210    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"openshift-service-ca.crt\""
Jul 26 18:59:00.105344 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.105239    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c75d5a\"/\"default-dockercfg-w44r6\""
Jul 26 18:59:00.114544 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.114525    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"]
Jul 26 18:59:00.193239 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.193210    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-model-cache\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.193239 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.193253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-dshm\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.193452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.193270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-tmp-dir\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.193452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.193358    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftshq\" (UniqueName: \"kubernetes.io/projected/08536619-6ad1-4018-9ead-083241a4c2d6-kube-api-access-ftshq\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.193452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.193403    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-home\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.193452 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.193427    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08536619-6ad1-4018-9ead-083241a4c2d6-tls-certs\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.294183 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.294141    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-model-cache\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.294359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.294203    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-dshm\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.294359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.294229    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-tmp-dir\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.294359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.294277    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftshq\" (UniqueName: \"kubernetes.io/projected/08536619-6ad1-4018-9ead-083241a4c2d6-kube-api-access-ftshq\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.294359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.294318    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-home\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.294359 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.294348    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08536619-6ad1-4018-9ead-083241a4c2d6-tls-certs\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.294612 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.294534    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-model-cache\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.294709 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.294644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-tmp-dir\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.294773 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.294726    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-home\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.296608 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.296586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-dshm\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.296848 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.296831    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08536619-6ad1-4018-9ead-083241a4c2d6-tls-certs\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.303045 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.303024    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftshq\" (UniqueName: \"kubernetes.io/projected/08536619-6ad1-4018-9ead-083241a4c2d6-kube-api-access-ftshq\") pod \"rate-test-model-c75d5a-kserve-65686d89f5-7ck65\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") " pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.412335 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.412242    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:00.540481 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.540457    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"]
Jul 26 18:59:00.542491 ip-10-0-138-70 kubenswrapper[2559]: W0726 18:59:00.542457    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08536619_6ad1_4018_9ead_083241a4c2d6.slice/crio-60318f22180b866ffc6b31123db1f2d2b799218b1a49364d83764a2badf6ac05 WatchSource:0}: Error finding container 60318f22180b866ffc6b31123db1f2d2b799218b1a49364d83764a2badf6ac05: Status 404 returned error can't find the container with id 60318f22180b866ffc6b31123db1f2d2b799218b1a49364d83764a2badf6ac05
Jul 26 18:59:00.927308 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.927264    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65" event={"ID":"08536619-6ad1-4018-9ead-083241a4c2d6","Type":"ContainerStarted","Data":"12db23b4b17afb7ca751cf2a4622a3850a9810299119945e3599b67ad3bada75"}
Jul 26 18:59:00.927308 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.927313    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65" event={"ID":"08536619-6ad1-4018-9ead-083241a4c2d6","Type":"ContainerStarted","Data":"60318f22180b866ffc6b31123db1f2d2b799218b1a49364d83764a2badf6ac05"}
Jul 26 18:59:00.945515 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:00.945462    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65" podStartSLOduration=0.945445688 podStartE2EDuration="945.445688ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 18:59:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 18:59:00.943376862 +0000 UTC m=+2812.267898959" watchObservedRunningTime="2026-07-26 18:59:00.945445688 +0000 UTC m=+2812.269967786"
Jul 26 18:59:10.412794 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:10.412758    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:10.413225 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:10.412814    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:10.425107 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:10.425083    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 18:59:10.988441 ip-10-0-138-70 kubenswrapper[2559]: I0726 18:59:10.988411    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 19:00:10.159867 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:10.159766    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"]
Jul 26 19:00:11.141767 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:11.141727    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"]
Jul 26 19:00:25.179939 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.179885    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz" podUID="60b36419-e171-402f-bb44-2e1234ce56d1" containerName="main" containerID="cri-o://cca03ea0284d14d6f6caa9e5444dd41cf36fab5d3b3f606c92d0ff539ca26443" gracePeriod=45
Jul 26 19:00:25.353212 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.353177    2559 generic.go:356] "Generic (PLEG): container finished" podID="60b36419-e171-402f-bb44-2e1234ce56d1" containerID="cca03ea0284d14d6f6caa9e5444dd41cf36fab5d3b3f606c92d0ff539ca26443" exitCode=0
Jul 26 19:00:25.353335 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.353237    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz" event={"ID":"60b36419-e171-402f-bb44-2e1234ce56d1","Type":"ContainerDied","Data":"cca03ea0284d14d6f6caa9e5444dd41cf36fab5d3b3f606c92d0ff539ca26443"}
Jul 26 19:00:25.433117 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.433055    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 19:00:25.587746 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.587718    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-home\") pod \"60b36419-e171-402f-bb44-2e1234ce56d1\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") "
Jul 26 19:00:25.587916 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.587787    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b36419-e171-402f-bb44-2e1234ce56d1-tls-certs\") pod \"60b36419-e171-402f-bb44-2e1234ce56d1\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") "
Jul 26 19:00:25.587916 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.587807    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-model-cache\") pod \"60b36419-e171-402f-bb44-2e1234ce56d1\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") "
Jul 26 19:00:25.587916 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.587838    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-dshm\") pod \"60b36419-e171-402f-bb44-2e1234ce56d1\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") "
Jul 26 19:00:25.587916 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.587856    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25kph\" (UniqueName: \"kubernetes.io/projected/60b36419-e171-402f-bb44-2e1234ce56d1-kube-api-access-25kph\") pod \"60b36419-e171-402f-bb44-2e1234ce56d1\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") "
Jul 26 19:00:25.587916 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.587887    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-tmp-dir\") pod \"60b36419-e171-402f-bb44-2e1234ce56d1\" (UID: \"60b36419-e171-402f-bb44-2e1234ce56d1\") "
Jul 26 19:00:25.588186 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.588015    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-home" (OuterVolumeSpecName: "home") pod "60b36419-e171-402f-bb44-2e1234ce56d1" (UID: "60b36419-e171-402f-bb44-2e1234ce56d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:00:25.588186 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.588093    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "60b36419-e171-402f-bb44-2e1234ce56d1" (UID: "60b36419-e171-402f-bb44-2e1234ce56d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:00:25.588294 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.588194    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "60b36419-e171-402f-bb44-2e1234ce56d1" (UID: "60b36419-e171-402f-bb44-2e1234ce56d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:00:25.588294 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.588261    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:25.588294 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.588276    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:25.589838 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.589814    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-dshm" (OuterVolumeSpecName: "dshm") pod "60b36419-e171-402f-bb44-2e1234ce56d1" (UID: "60b36419-e171-402f-bb44-2e1234ce56d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:00:25.589917 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.589849    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60b36419-e171-402f-bb44-2e1234ce56d1-kube-api-access-25kph" (OuterVolumeSpecName: "kube-api-access-25kph") pod "60b36419-e171-402f-bb44-2e1234ce56d1" (UID: "60b36419-e171-402f-bb44-2e1234ce56d1"). InnerVolumeSpecName "kube-api-access-25kph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:00:25.589964 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.589917    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60b36419-e171-402f-bb44-2e1234ce56d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "60b36419-e171-402f-bb44-2e1234ce56d1" (UID: "60b36419-e171-402f-bb44-2e1234ce56d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 19:00:25.689547 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.689484    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:25.689547 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.689507    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60b36419-e171-402f-bb44-2e1234ce56d1-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:25.689547 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.689515    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60b36419-e171-402f-bb44-2e1234ce56d1-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:25.689547 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:25.689525    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25kph\" (UniqueName: \"kubernetes.io/projected/60b36419-e171-402f-bb44-2e1234ce56d1-kube-api-access-25kph\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:26.160637 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.160524    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65" podUID="08536619-6ad1-4018-9ead-083241a4c2d6" containerName="main" containerID="cri-o://12db23b4b17afb7ca751cf2a4622a3850a9810299119945e3599b67ad3bada75" gracePeriod=45
Jul 26 19:00:26.361350 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.361321    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"
Jul 26 19:00:26.361809 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.361346    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz" event={"ID":"60b36419-e171-402f-bb44-2e1234ce56d1","Type":"ContainerDied","Data":"8114daf96cad00e519fdbc7e2bf43d5f7e6713857d7aace6fe63a10b1052d3c3"}
Jul 26 19:00:26.361809 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.361419    2559 scope.go:117] "RemoveContainer" containerID="cca03ea0284d14d6f6caa9e5444dd41cf36fab5d3b3f606c92d0ff539ca26443"
Jul 26 19:00:26.363733 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.363698    2559 generic.go:356] "Generic (PLEG): container finished" podID="08536619-6ad1-4018-9ead-083241a4c2d6" containerID="12db23b4b17afb7ca751cf2a4622a3850a9810299119945e3599b67ad3bada75" exitCode=0
Jul 26 19:00:26.363854 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.363737    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65" event={"ID":"08536619-6ad1-4018-9ead-083241a4c2d6","Type":"ContainerDied","Data":"12db23b4b17afb7ca751cf2a4622a3850a9810299119945e3599b67ad3bada75"}
Jul 26 19:00:26.385123 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.385100    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"]
Jul 26 19:00:26.389478 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.389459    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2b193e/rate-test-model-2b193e-kserve-c8d57fff-zzjbz"]
Jul 26 19:00:26.412853 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.412836    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 19:00:26.497638 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.497591    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-tmp-dir\") pod \"08536619-6ad1-4018-9ead-083241a4c2d6\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") "
Jul 26 19:00:26.497857 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.497661    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftshq\" (UniqueName: \"kubernetes.io/projected/08536619-6ad1-4018-9ead-083241a4c2d6-kube-api-access-ftshq\") pod \"08536619-6ad1-4018-9ead-083241a4c2d6\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") "
Jul 26 19:00:26.497857 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.497711    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-dshm\") pod \"08536619-6ad1-4018-9ead-083241a4c2d6\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") "
Jul 26 19:00:26.497857 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.497729    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-model-cache\") pod \"08536619-6ad1-4018-9ead-083241a4c2d6\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") "
Jul 26 19:00:26.497857 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.497779    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08536619-6ad1-4018-9ead-083241a4c2d6-tls-certs\") pod \"08536619-6ad1-4018-9ead-083241a4c2d6\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") "
Jul 26 19:00:26.497857 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.497797    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-home\") pod \"08536619-6ad1-4018-9ead-083241a4c2d6\" (UID: \"08536619-6ad1-4018-9ead-083241a4c2d6\") "
Jul 26 19:00:26.498133 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.497873    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "08536619-6ad1-4018-9ead-083241a4c2d6" (UID: "08536619-6ad1-4018-9ead-083241a4c2d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:00:26.498133 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.498048    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "08536619-6ad1-4018-9ead-083241a4c2d6" (UID: "08536619-6ad1-4018-9ead-083241a4c2d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:00:26.498232 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.498138    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-home" (OuterVolumeSpecName: "home") pod "08536619-6ad1-4018-9ead-083241a4c2d6" (UID: "08536619-6ad1-4018-9ead-083241a4c2d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:00:26.498232 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.498222    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:26.498308 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.498241    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:26.498308 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.498255    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:26.499836 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.499818    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-dshm" (OuterVolumeSpecName: "dshm") pod "08536619-6ad1-4018-9ead-083241a4c2d6" (UID: "08536619-6ad1-4018-9ead-083241a4c2d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:00:26.499924 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.499909    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08536619-6ad1-4018-9ead-083241a4c2d6-kube-api-access-ftshq" (OuterVolumeSpecName: "kube-api-access-ftshq") pod "08536619-6ad1-4018-9ead-083241a4c2d6" (UID: "08536619-6ad1-4018-9ead-083241a4c2d6"). InnerVolumeSpecName "kube-api-access-ftshq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:00:26.499985 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.499971    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08536619-6ad1-4018-9ead-083241a4c2d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "08536619-6ad1-4018-9ead-083241a4c2d6" (UID: "08536619-6ad1-4018-9ead-083241a4c2d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 19:00:26.599020 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.598985    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftshq\" (UniqueName: \"kubernetes.io/projected/08536619-6ad1-4018-9ead-083241a4c2d6-kube-api-access-ftshq\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:26.599020 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.599014    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08536619-6ad1-4018-9ead-083241a4c2d6-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:26.599200 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:26.599026    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08536619-6ad1-4018-9ead-083241a4c2d6-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:00:27.176850 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:27.176814    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60b36419-e171-402f-bb44-2e1234ce56d1" path="/var/lib/kubelet/pods/60b36419-e171-402f-bb44-2e1234ce56d1/volumes"
Jul 26 19:00:27.370867 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:27.370837    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65" event={"ID":"08536619-6ad1-4018-9ead-083241a4c2d6","Type":"ContainerDied","Data":"60318f22180b866ffc6b31123db1f2d2b799218b1a49364d83764a2badf6ac05"}
Jul 26 19:00:27.370867 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:27.370858    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"
Jul 26 19:00:27.371378 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:27.370891    2559 scope.go:117] "RemoveContainer" containerID="12db23b4b17afb7ca751cf2a4622a3850a9810299119945e3599b67ad3bada75"
Jul 26 19:00:27.389195 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:27.389172    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"]
Jul 26 19:00:27.393329 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:27.393307    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c75d5a/rate-test-model-c75d5a-kserve-65686d89f5-7ck65"]
Jul 26 19:00:29.178063 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:29.178030    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08536619-6ad1-4018-9ead-083241a4c2d6" path="/var/lib/kubelet/pods/08536619-6ad1-4018-9ead-083241a4c2d6/volumes"
Jul 26 19:00:34.321742 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.321713    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"]
Jul 26 19:00:34.322237 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.322122    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08536619-6ad1-4018-9ead-083241a4c2d6" containerName="main"
Jul 26 19:00:34.322237 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.322133    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="08536619-6ad1-4018-9ead-083241a4c2d6" containerName="main"
Jul 26 19:00:34.322237 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.322156    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60b36419-e171-402f-bb44-2e1234ce56d1" containerName="main"
Jul 26 19:00:34.322237 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.322161    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="60b36419-e171-402f-bb44-2e1234ce56d1" containerName="main"
Jul 26 19:00:34.322423 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.322244    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="08536619-6ad1-4018-9ead-083241a4c2d6" containerName="main"
Jul 26 19:00:34.322423 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.322256    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="60b36419-e171-402f-bb44-2e1234ce56d1" containerName="main"
Jul 26 19:00:34.326737 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.326713    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:00:34.330026 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.330002    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-4735db\""
Jul 26 19:00:34.330353 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.330339    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-4735db-dockercfg-dppb2\""
Jul 26 19:00:34.346761 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.346740    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"]
Jul 26 19:00:34.353777 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.353753    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"]
Jul 26 19:00:34.357272 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.357256    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:34.367320 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.367301    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w98pj\" (UniqueName: \"kubernetes.io/projected/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-kube-api-access-w98pj\") pod \"payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565\" (UID: \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:00:34.367417 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.367343    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565\" (UID: \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:00:34.367417 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.367373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7885\" (UniqueName: \"kubernetes.io/projected/05b08811-4f6e-496a-af71-e44bcc8d5887-kube-api-access-k7885\") pod \"payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s\" (UID: \"05b08811-4f6e-496a-af71-e44bcc8d5887\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:34.367493 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.367442    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05b08811-4f6e-496a-af71-e44bcc8d5887-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s\" (UID: \"05b08811-4f6e-496a-af71-e44bcc8d5887\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:34.372202 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.372181    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"]
Jul 26 19:00:34.468431 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.468402    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05b08811-4f6e-496a-af71-e44bcc8d5887-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s\" (UID: \"05b08811-4f6e-496a-af71-e44bcc8d5887\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:34.468580 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.468456    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w98pj\" (UniqueName: \"kubernetes.io/projected/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-kube-api-access-w98pj\") pod \"payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565\" (UID: \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:00:34.468580 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.468507    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565\" (UID: \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:00:34.468580 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.468539    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7885\" (UniqueName: \"kubernetes.io/projected/05b08811-4f6e-496a-af71-e44bcc8d5887-kube-api-access-k7885\") pod \"payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s\" (UID: \"05b08811-4f6e-496a-af71-e44bcc8d5887\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:34.469124 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.469104    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05b08811-4f6e-496a-af71-e44bcc8d5887-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s\" (UID: \"05b08811-4f6e-496a-af71-e44bcc8d5887\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:34.469250 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.469233    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565\" (UID: \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:00:34.477108 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.477080    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7885\" (UniqueName: \"kubernetes.io/projected/05b08811-4f6e-496a-af71-e44bcc8d5887-kube-api-access-k7885\") pod \"payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s\" (UID: \"05b08811-4f6e-496a-af71-e44bcc8d5887\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:34.477221 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.477180    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w98pj\" (UniqueName: \"kubernetes.io/projected/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-kube-api-access-w98pj\") pod \"payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565\" (UID: \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:00:34.638301 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.638215    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:00:34.667629 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.667588    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:34.787756 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.787726    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"]
Jul 26 19:00:34.790003 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:00:34.789974    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb24bfc_b62c_4bb8_bce5_6f9e9b5b3a7d.slice/crio-c2b256a821704764c8a8aaafd7822ea4a1799af6c6afa4dd759f806ed255b440 WatchSource:0}: Error finding container c2b256a821704764c8a8aaafd7822ea4a1799af6c6afa4dd759f806ed255b440: Status 404 returned error can't find the container with id c2b256a821704764c8a8aaafd7822ea4a1799af6c6afa4dd759f806ed255b440
Jul 26 19:00:34.817960 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:34.817935    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"]
Jul 26 19:00:34.819939 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:00:34.819901    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b08811_4f6e_496a_af71_e44bcc8d5887.slice/crio-49bf3aae4fd2f87148f86058371143899b183006ce0a04e992a25ee24277e74f WatchSource:0}: Error finding container 49bf3aae4fd2f87148f86058371143899b183006ce0a04e992a25ee24277e74f: Status 404 returned error can't find the container with id 49bf3aae4fd2f87148f86058371143899b183006ce0a04e992a25ee24277e74f
Jul 26 19:00:35.415197 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:35.415159    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565" event={"ID":"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d","Type":"ContainerStarted","Data":"f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71"}
Jul 26 19:00:35.415652 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:35.415217    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565" event={"ID":"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d","Type":"ContainerStarted","Data":"c2b256a821704764c8a8aaafd7822ea4a1799af6c6afa4dd759f806ed255b440"}
Jul 26 19:00:35.415652 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:35.415335    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:00:35.416691 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:35.416669    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s" event={"ID":"05b08811-4f6e-496a-af71-e44bcc8d5887","Type":"ContainerStarted","Data":"f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925"}
Jul 26 19:00:35.416810 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:35.416709    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:35.416810 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:35.416726    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s" event={"ID":"05b08811-4f6e-496a-af71-e44bcc8d5887","Type":"ContainerStarted","Data":"49bf3aae4fd2f87148f86058371143899b183006ce0a04e992a25ee24277e74f"}
Jul 26 19:00:35.435724 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:35.435682    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565" podStartSLOduration=1.43566838 podStartE2EDuration="1.43566838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:00:35.432664447 +0000 UTC m=+2906.757186541" watchObservedRunningTime="2026-07-26 19:00:35.43566838 +0000 UTC m=+2906.760190477"
Jul 26 19:00:35.449413 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:35.449369    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s" podStartSLOduration=1.449357904 podStartE2EDuration="1.449357904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:00:35.447554714 +0000 UTC m=+2906.772076807" watchObservedRunningTime="2026-07-26 19:00:35.449357904 +0000 UTC m=+2906.773880001"
Jul 26 19:00:46.430454 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:46.430411    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565" podUID="ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.69:9004: connect: connection refused"
Jul 26 19:00:46.431007 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:46.430989    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:00:56.430201 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:00:56.430163    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:01:03.504107 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.504072    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"]
Jul 26 19:01:03.507817 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.507797    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:03.510785 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.510762    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-1318a7-dockercfg-jz4z8\""
Jul 26 19:01:03.510917 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.510893    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-1318a7\""
Jul 26 19:01:03.517986 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.517960    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"]
Jul 26 19:01:03.535515 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.535483    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"]
Jul 26 19:01:03.539181 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.539155    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:03.551320 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.551300    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"]
Jul 26 19:01:03.647210 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.647177    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sj64\" (UniqueName: \"kubernetes.io/projected/1c274fd7-320f-4ad2-b792-f1f38fa9128a-kube-api-access-2sj64\") pod \"payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv\" (UID: \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:03.647210 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.647212    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79f61a20-c303-4325-848c-e3c9764701eb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr\" (UID: \"79f61a20-c303-4325-848c-e3c9764701eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:03.647476 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.647231    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfpp7\" (UniqueName: \"kubernetes.io/projected/79f61a20-c303-4325-848c-e3c9764701eb-kube-api-access-bfpp7\") pod \"payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr\" (UID: \"79f61a20-c303-4325-848c-e3c9764701eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:03.647476 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.647306    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c274fd7-320f-4ad2-b792-f1f38fa9128a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv\" (UID: \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:03.748936 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.748896    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c274fd7-320f-4ad2-b792-f1f38fa9128a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv\" (UID: \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:03.749126 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.749090    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sj64\" (UniqueName: \"kubernetes.io/projected/1c274fd7-320f-4ad2-b792-f1f38fa9128a-kube-api-access-2sj64\") pod \"payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv\" (UID: \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:03.749126 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.749116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79f61a20-c303-4325-848c-e3c9764701eb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr\" (UID: \"79f61a20-c303-4325-848c-e3c9764701eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:03.749249 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.749140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfpp7\" (UniqueName: \"kubernetes.io/projected/79f61a20-c303-4325-848c-e3c9764701eb-kube-api-access-bfpp7\") pod \"payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr\" (UID: \"79f61a20-c303-4325-848c-e3c9764701eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:03.749717 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.749696    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c274fd7-320f-4ad2-b792-f1f38fa9128a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv\" (UID: \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:03.749838 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.749821    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79f61a20-c303-4325-848c-e3c9764701eb-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr\" (UID: \"79f61a20-c303-4325-848c-e3c9764701eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:03.756898 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.756835    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sj64\" (UniqueName: \"kubernetes.io/projected/1c274fd7-320f-4ad2-b792-f1f38fa9128a-kube-api-access-2sj64\") pod \"payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv\" (UID: \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:03.757007 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.756903    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfpp7\" (UniqueName: \"kubernetes.io/projected/79f61a20-c303-4325-848c-e3c9764701eb-kube-api-access-bfpp7\") pod \"payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr\" (UID: \"79f61a20-c303-4325-848c-e3c9764701eb\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:03.819877 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.819854    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:03.850628 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.850590    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:03.970340 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:03.970217    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"]
Jul 26 19:01:03.973959 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:01:03.973926    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c274fd7_320f_4ad2_b792_f1f38fa9128a.slice/crio-442ebe066c9bfc535eaecf30c3a65d8cb38ac9fe73f080cbbca847acaa70faf4 WatchSource:0}: Error finding container 442ebe066c9bfc535eaecf30c3a65d8cb38ac9fe73f080cbbca847acaa70faf4: Status 404 returned error can't find the container with id 442ebe066c9bfc535eaecf30c3a65d8cb38ac9fe73f080cbbca847acaa70faf4
Jul 26 19:01:04.006164 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:04.006123    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"]
Jul 26 19:01:04.009285 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:01:04.009217    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f61a20_c303_4325_848c_e3c9764701eb.slice/crio-bda637f2f08d38150f23debbb4879b3afe15946038d1ad6ec89eeff04db0f7b0 WatchSource:0}: Error finding container bda637f2f08d38150f23debbb4879b3afe15946038d1ad6ec89eeff04db0f7b0: Status 404 returned error can't find the container with id bda637f2f08d38150f23debbb4879b3afe15946038d1ad6ec89eeff04db0f7b0
Jul 26 19:01:04.577308 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:04.577269    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv" event={"ID":"1c274fd7-320f-4ad2-b792-f1f38fa9128a","Type":"ContainerStarted","Data":"4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648"}
Jul 26 19:01:04.577779 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:04.577328    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv" event={"ID":"1c274fd7-320f-4ad2-b792-f1f38fa9128a","Type":"ContainerStarted","Data":"442ebe066c9bfc535eaecf30c3a65d8cb38ac9fe73f080cbbca847acaa70faf4"}
Jul 26 19:01:04.577779 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:04.577409    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:04.578910 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:04.578891    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr" event={"ID":"79f61a20-c303-4325-848c-e3c9764701eb","Type":"ContainerStarted","Data":"aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730"}
Jul 26 19:01:04.578966 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:04.578927    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr" event={"ID":"79f61a20-c303-4325-848c-e3c9764701eb","Type":"ContainerStarted","Data":"bda637f2f08d38150f23debbb4879b3afe15946038d1ad6ec89eeff04db0f7b0"}
Jul 26 19:01:04.579004 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:04.578960    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:04.594342 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:04.594291    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv" podStartSLOduration=1.594278662 podStartE2EDuration="1.594278662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:01:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:01:04.592794529 +0000 UTC m=+2935.917316627" watchObservedRunningTime="2026-07-26 19:01:04.594278662 +0000 UTC m=+2935.918800758"
Jul 26 19:01:04.608072 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:04.608033    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr" podStartSLOduration=1.6080203339999999 podStartE2EDuration="1.608020334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:01:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:01:04.607422443 +0000 UTC m=+2935.931944544" watchObservedRunningTime="2026-07-26 19:01:04.608020334 +0000 UTC m=+2935.932542432"
Jul 26 19:01:15.585508 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:15.585478    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:01:15.585958 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:15.585546    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:01:24.280590 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:24.280556    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr_79f61a20-c303-4325-848c-e3c9764701eb/payload-processing/0.log"
Jul 26 19:01:27.701145 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.701117    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"]
Jul 26 19:01:27.705087 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.705068    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.709329 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.709308    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-4735db\"/\"kube-root-ca.crt\""
Jul 26 19:01:27.709329 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.709320    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-4735db\"/\"openshift-service-ca.crt\""
Jul 26 19:01:27.709516 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.709380    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-4735db\"/\"ipp-route-4735db-kserve-self-signed-certs\""
Jul 26 19:01:27.709516 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.709409    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-4735db\"/\"default-dockercfg-672q8\""
Jul 26 19:01:27.713552 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.713527    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"]
Jul 26 19:01:27.769873 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.769848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-dshm\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.770022 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.769896    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30228679-2f7e-47bc-80a1-94ed89c06ebf-tls-certs\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.770022 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.769920    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-tmp-dir\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.770022 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.769977    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-model-cache\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.770022 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.770010    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbxkh\" (UniqueName: \"kubernetes.io/projected/30228679-2f7e-47bc-80a1-94ed89c06ebf-kube-api-access-jbxkh\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.770181 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.770038    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-home\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.871489 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.871456    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-dshm\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.871693 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.871509    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30228679-2f7e-47bc-80a1-94ed89c06ebf-tls-certs\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.871693 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.871539    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-tmp-dir\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.871693 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.871568    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-model-cache\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.871693 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.871586    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbxkh\" (UniqueName: \"kubernetes.io/projected/30228679-2f7e-47bc-80a1-94ed89c06ebf-kube-api-access-jbxkh\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.871930 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.871766    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-home\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.872011 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.871979    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-tmp-dir\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.872055 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.872014    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-model-cache\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.872144 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.872107    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-home\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.873865 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.873842    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-dshm\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.874011 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.873994    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30228679-2f7e-47bc-80a1-94ed89c06ebf-tls-certs\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:27.879665 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:27.879644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbxkh\" (UniqueName: \"kubernetes.io/projected/30228679-2f7e-47bc-80a1-94ed89c06ebf-kube-api-access-jbxkh\") pod \"ipp-route-4735db-kserve-56cb5998c5-7jpxh\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") " pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:28.016274 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:28.016192    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:28.162414 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:28.162385    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"]
Jul 26 19:01:28.164444 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:01:28.164414    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30228679_2f7e_47bc_80a1_94ed89c06ebf.slice/crio-a024b0ba37fb563bfa595784536a952acfe905510c9f880b4cdc075a8df974b8 WatchSource:0}: Error finding container a024b0ba37fb563bfa595784536a952acfe905510c9f880b4cdc075a8df974b8: Status 404 returned error can't find the container with id a024b0ba37fb563bfa595784536a952acfe905510c9f880b4cdc075a8df974b8
Jul 26 19:01:28.702381 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:28.702350    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh" event={"ID":"30228679-2f7e-47bc-80a1-94ed89c06ebf","Type":"ContainerStarted","Data":"2cab069b0b9aafe996cb728eabd11d45a703b362423e0a95950494258ccdfd4b"}
Jul 26 19:01:28.702802 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:28.702392    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh" event={"ID":"30228679-2f7e-47bc-80a1-94ed89c06ebf","Type":"ContainerStarted","Data":"a024b0ba37fb563bfa595784536a952acfe905510c9f880b4cdc075a8df974b8"}
Jul 26 19:01:28.719105 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:28.719060    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh" podStartSLOduration=1.719043348 podStartE2EDuration="1.719043348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:01:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:01:28.71778204 +0000 UTC m=+2960.042304137" watchObservedRunningTime="2026-07-26 19:01:28.719043348 +0000 UTC m=+2960.043565476"
Jul 26 19:01:38.016683 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:38.016565    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:38.017056 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:38.016684    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:38.029279 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:38.029254    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:01:38.773167 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:01:38.773137    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:02:10.015404 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:10.015282    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 19:02:10.027741 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:10.027719    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7pqql_d640dee1-af4a-450f-8d66-924e7d08dd98/ovn-acl-logging/0.log"
Jul 26 19:02:11.035716 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:11.035672    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s_05b08811-4f6e-496a-af71-e44bcc8d5887/payload-processing/0.log"
Jul 26 19:02:11.289745 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:11.289659    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr_79f61a20-c303-4325-848c-e3c9764701eb/payload-processing/0.log"
Jul 26 19:02:15.582172 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.582141    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"]
Jul 26 19:02:15.585898 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.585876    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:15.588914 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.588888    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-9eba87\""
Jul 26 19:02:15.589054 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.588923    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-9eba87-dockercfg-8brjk\""
Jul 26 19:02:15.596117 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.596091    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"]
Jul 26 19:02:15.620894 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.620861    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwz2d\" (UniqueName: \"kubernetes.io/projected/ab1de98b-cd4a-4df9-b299-79c781a143a4-kube-api-access-hwz2d\") pod \"payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf\" (UID: \"ab1de98b-cd4a-4df9-b299-79c781a143a4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:15.621122 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.621086    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1de98b-cd4a-4df9-b299-79c781a143a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf\" (UID: \"ab1de98b-cd4a-4df9-b299-79c781a143a4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:15.721689 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.721660    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1de98b-cd4a-4df9-b299-79c781a143a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf\" (UID: \"ab1de98b-cd4a-4df9-b299-79c781a143a4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:15.721858 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.721703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwz2d\" (UniqueName: \"kubernetes.io/projected/ab1de98b-cd4a-4df9-b299-79c781a143a4-kube-api-access-hwz2d\") pod \"payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf\" (UID: \"ab1de98b-cd4a-4df9-b299-79c781a143a4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:15.722425 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.722404    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1de98b-cd4a-4df9-b299-79c781a143a4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf\" (UID: \"ab1de98b-cd4a-4df9-b299-79c781a143a4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:15.730013 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.729992    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwz2d\" (UniqueName: \"kubernetes.io/projected/ab1de98b-cd4a-4df9-b299-79c781a143a4-kube-api-access-hwz2d\") pod \"payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf\" (UID: \"ab1de98b-cd4a-4df9-b299-79c781a143a4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:15.898290 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:15.898207    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:16.036059 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:16.036036    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"]
Jul 26 19:02:16.038915 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:02:16.038850    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab1de98b_cd4a_4df9_b299_79c781a143a4.slice/crio-cccf81b79085c60e451311cff2cebed8278e42c9f10a54591307bbc1690b4c4d WatchSource:0}: Error finding container cccf81b79085c60e451311cff2cebed8278e42c9f10a54591307bbc1690b4c4d: Status 404 returned error can't find the container with id cccf81b79085c60e451311cff2cebed8278e42c9f10a54591307bbc1690b4c4d
Jul 26 19:02:16.967274 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:16.967242    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf" event={"ID":"ab1de98b-cd4a-4df9-b299-79c781a143a4","Type":"ContainerStarted","Data":"20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c"}
Jul 26 19:02:16.967744 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:16.967294    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf" event={"ID":"ab1de98b-cd4a-4df9-b299-79c781a143a4","Type":"ContainerStarted","Data":"cccf81b79085c60e451311cff2cebed8278e42c9f10a54591307bbc1690b4c4d"}
Jul 26 19:02:16.967744 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:16.967323    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:16.986042 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:16.985991    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf" podStartSLOduration=1.9859770110000001 podStartE2EDuration="1.985977011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:02:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:02:16.983492417 +0000 UTC m=+3008.308014515" watchObservedRunningTime="2026-07-26 19:02:16.985977011 +0000 UTC m=+3008.310499146"
Jul 26 19:02:27.974473 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:27.974443    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:29.983492 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:29.983457    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"]
Jul 26 19:02:29.987032 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:29.987016    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"
Jul 26 19:02:29.989889 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:29.989867    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 19:02:29.990002 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:29.989869    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 19:02:29.994495 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:29.994303    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"]
Jul 26 19:02:30.159366 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:30.159333    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5269594f-9a54-42c4-8429-9761aa2b088f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t\" (UID: \"5269594f-9a54-42c4-8429-9761aa2b088f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"
Jul 26 19:02:30.260537 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:30.260457    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5269594f-9a54-42c4-8429-9761aa2b088f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t\" (UID: \"5269594f-9a54-42c4-8429-9761aa2b088f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"
Jul 26 19:02:30.261064 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:30.261045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5269594f-9a54-42c4-8429-9761aa2b088f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t\" (UID: \"5269594f-9a54-42c4-8429-9761aa2b088f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"
Jul 26 19:02:30.299203 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:30.299177    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"
Jul 26 19:02:30.441727 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:30.441693    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"]
Jul 26 19:02:30.442866 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:02:30.442821    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5269594f_9a54_42c4_8429_9761aa2b088f.slice/crio-a8152433ded41d5a13f1fd16ae0b2aa10d8147d8dc7320df66cfe88aabbb2def WatchSource:0}: Error finding container a8152433ded41d5a13f1fd16ae0b2aa10d8147d8dc7320df66cfe88aabbb2def: Status 404 returned error can't find the container with id a8152433ded41d5a13f1fd16ae0b2aa10d8147d8dc7320df66cfe88aabbb2def
Jul 26 19:02:31.047252 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:31.047200    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t" event={"ID":"5269594f-9a54-42c4-8429-9761aa2b088f","Type":"ContainerStarted","Data":"ef65e18d6db539fa55817f42d97189673eb12ec7a25cdb68a45aa84bc1f3c7c8"}
Jul 26 19:02:31.047744 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:31.047315    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t" event={"ID":"5269594f-9a54-42c4-8429-9761aa2b088f","Type":"ContainerStarted","Data":"a8152433ded41d5a13f1fd16ae0b2aa10d8147d8dc7320df66cfe88aabbb2def"}
Jul 26 19:02:31.069694 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:31.069649    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t" podStartSLOduration=2.069634776 podStartE2EDuration="2.069634776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:02:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:02:31.067371885 +0000 UTC m=+3022.391893982" watchObservedRunningTime="2026-07-26 19:02:31.069634776 +0000 UTC m=+3022.394156870"
Jul 26 19:02:32.053726 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:32.053688    2559 generic.go:356] "Generic (PLEG): container finished" podID="5269594f-9a54-42c4-8429-9761aa2b088f" containerID="ef65e18d6db539fa55817f42d97189673eb12ec7a25cdb68a45aa84bc1f3c7c8" exitCode=0
Jul 26 19:02:32.054134 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:32.053781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t" event={"ID":"5269594f-9a54-42c4-8429-9761aa2b088f","Type":"ContainerDied","Data":"ef65e18d6db539fa55817f42d97189673eb12ec7a25cdb68a45aa84bc1f3c7c8"}
Jul 26 19:02:33.197393 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:33.197372    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"
Jul 26 19:02:33.289417 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:33.289386    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5269594f-9a54-42c4-8429-9761aa2b088f-maas-api-service-ca\") pod \"5269594f-9a54-42c4-8429-9761aa2b088f\" (UID: \"5269594f-9a54-42c4-8429-9761aa2b088f\") "
Jul 26 19:02:33.289759 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:33.289733    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5269594f-9a54-42c4-8429-9761aa2b088f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5269594f-9a54-42c4-8429-9761aa2b088f" (UID: "5269594f-9a54-42c4-8429-9761aa2b088f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:02:33.390525 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:33.390448    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5269594f-9a54-42c4-8429-9761aa2b088f-maas-api-service-ca\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:02:34.066070 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:34.066043    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"
Jul 26 19:02:34.066226 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:34.066075    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t" event={"ID":"5269594f-9a54-42c4-8429-9761aa2b088f","Type":"ContainerDied","Data":"a8152433ded41d5a13f1fd16ae0b2aa10d8147d8dc7320df66cfe88aabbb2def"}
Jul 26 19:02:34.066226 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:34.066115    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8152433ded41d5a13f1fd16ae0b2aa10d8147d8dc7320df66cfe88aabbb2def"
Jul 26 19:02:40.445097 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:40.445060    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"]
Jul 26 19:02:40.445603 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:40.445335    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf" podUID="ab1de98b-cd4a-4df9-b299-79c781a143a4" containerName="payload-pre-processing" containerID="cri-o://20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c" gracePeriod=30
Jul 26 19:02:40.704184 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:40.704128    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:40.760598 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:40.760565    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1de98b-cd4a-4df9-b299-79c781a143a4-plugins-config-volume\") pod \"ab1de98b-cd4a-4df9-b299-79c781a143a4\" (UID: \"ab1de98b-cd4a-4df9-b299-79c781a143a4\") "
Jul 26 19:02:40.760749 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:40.760665    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwz2d\" (UniqueName: \"kubernetes.io/projected/ab1de98b-cd4a-4df9-b299-79c781a143a4-kube-api-access-hwz2d\") pod \"ab1de98b-cd4a-4df9-b299-79c781a143a4\" (UID: \"ab1de98b-cd4a-4df9-b299-79c781a143a4\") "
Jul 26 19:02:40.760990 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:40.760959    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab1de98b-cd4a-4df9-b299-79c781a143a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab1de98b-cd4a-4df9-b299-79c781a143a4" (UID: "ab1de98b-cd4a-4df9-b299-79c781a143a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:02:40.762650 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:40.762603    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab1de98b-cd4a-4df9-b299-79c781a143a4-kube-api-access-hwz2d" (OuterVolumeSpecName: "kube-api-access-hwz2d") pod "ab1de98b-cd4a-4df9-b299-79c781a143a4" (UID: "ab1de98b-cd4a-4df9-b299-79c781a143a4"). InnerVolumeSpecName "kube-api-access-hwz2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:02:40.861860 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:40.861834    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1de98b-cd4a-4df9-b299-79c781a143a4-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:02:40.861860 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:40.861856    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwz2d\" (UniqueName: \"kubernetes.io/projected/ab1de98b-cd4a-4df9-b299-79c781a143a4-kube-api-access-hwz2d\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:02:41.104199 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.104112    2559 generic.go:356] "Generic (PLEG): container finished" podID="ab1de98b-cd4a-4df9-b299-79c781a143a4" containerID="20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c" exitCode=0
Jul 26 19:02:41.104199 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.104159    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"
Jul 26 19:02:41.104387 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.104210    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf" event={"ID":"ab1de98b-cd4a-4df9-b299-79c781a143a4","Type":"ContainerDied","Data":"20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c"}
Jul 26 19:02:41.104387 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.104262    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf" event={"ID":"ab1de98b-cd4a-4df9-b299-79c781a143a4","Type":"ContainerDied","Data":"cccf81b79085c60e451311cff2cebed8278e42c9f10a54591307bbc1690b4c4d"}
Jul 26 19:02:41.104387 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.104286    2559 scope.go:117] "RemoveContainer" containerID="20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c"
Jul 26 19:02:41.116379 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.114783    2559 scope.go:117] "RemoveContainer" containerID="20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c"
Jul 26 19:02:41.116379 ip-10-0-138-70 kubenswrapper[2559]: E0726 19:02:41.116338    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c\": container with ID starting with 20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c not found: ID does not exist" containerID="20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c"
Jul 26 19:02:41.116488 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.116372    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c"} err="failed to get container status \"20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c\": rpc error: code = NotFound desc = could not find container \"20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c\": container with ID starting with 20b0b182ae0bb6ab7308c46a74e529e783a17332c79ae9d3270c6a232e8fc41c not found: ID does not exist"
Jul 26 19:02:41.126779 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.126757    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"]
Jul 26 19:02:41.130101 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.130078    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-9eba87-56b9cb4948-rd6zf"]
Jul 26 19:02:41.176392 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:41.176370    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab1de98b-cd4a-4df9-b299-79c781a143a4" path="/var/lib/kubelet/pods/ab1de98b-cd4a-4df9-b299-79c781a143a4/volumes"
Jul 26 19:02:45.060943 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:45.060910    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"]
Jul 26 19:02:45.063343 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:45.063319    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-9eba87-tsf9t"]
Jul 26 19:02:45.176598 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:02:45.176562    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5269594f-9a54-42c4-8429-9761aa2b088f" path="/var/lib/kubelet/pods/5269594f-9a54-42c4-8429-9761aa2b088f/volumes"
Jul 26 19:03:10.477273 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.477236    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"]
Jul 26 19:03:10.477771 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.477491    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s" podUID="05b08811-4f6e-496a-af71-e44bcc8d5887" containerName="payload-processing" containerID="cri-o://f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925" gracePeriod=30
Jul 26 19:03:10.484635 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.484591    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"]
Jul 26 19:03:10.484918 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.484863    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565" podUID="ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d" containerName="payload-pre-processing" containerID="cri-o://f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71" gracePeriod=30
Jul 26 19:03:10.746278 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.746211    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:03:10.772605 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.772572    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:03:10.861817 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.861789    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7885\" (UniqueName: \"kubernetes.io/projected/05b08811-4f6e-496a-af71-e44bcc8d5887-kube-api-access-k7885\") pod \"05b08811-4f6e-496a-af71-e44bcc8d5887\" (UID: \"05b08811-4f6e-496a-af71-e44bcc8d5887\") "
Jul 26 19:03:10.861974 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.861833    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05b08811-4f6e-496a-af71-e44bcc8d5887-plugins-config-volume\") pod \"05b08811-4f6e-496a-af71-e44bcc8d5887\" (UID: \"05b08811-4f6e-496a-af71-e44bcc8d5887\") "
Jul 26 19:03:10.862277 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.862251    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05b08811-4f6e-496a-af71-e44bcc8d5887-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "05b08811-4f6e-496a-af71-e44bcc8d5887" (UID: "05b08811-4f6e-496a-af71-e44bcc8d5887"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:03:10.863659 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.863637    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05b08811-4f6e-496a-af71-e44bcc8d5887-kube-api-access-k7885" (OuterVolumeSpecName: "kube-api-access-k7885") pod "05b08811-4f6e-496a-af71-e44bcc8d5887" (UID: "05b08811-4f6e-496a-af71-e44bcc8d5887"). InnerVolumeSpecName "kube-api-access-k7885". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:03:10.963346 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.963320    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w98pj\" (UniqueName: \"kubernetes.io/projected/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-kube-api-access-w98pj\") pod \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\" (UID: \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\") "
Jul 26 19:03:10.963486 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.963395    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-plugins-config-volume\") pod \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\" (UID: \"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d\") "
Jul 26 19:03:10.963745 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.963726    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7885\" (UniqueName: \"kubernetes.io/projected/05b08811-4f6e-496a-af71-e44bcc8d5887-kube-api-access-k7885\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:10.963803 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.963737    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d" (UID: "ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:03:10.963803 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.963750    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05b08811-4f6e-496a-af71-e44bcc8d5887-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:10.965459 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:10.965428    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-kube-api-access-w98pj" (OuterVolumeSpecName: "kube-api-access-w98pj") pod "ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d" (UID: "ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d"). InnerVolumeSpecName "kube-api-access-w98pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:03:11.064207 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.064139    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w98pj\" (UniqueName: \"kubernetes.io/projected/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-kube-api-access-w98pj\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:11.064207 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.064164    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:11.271480 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.271452    2559 generic.go:356] "Generic (PLEG): container finished" podID="05b08811-4f6e-496a-af71-e44bcc8d5887" containerID="f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925" exitCode=0
Jul 26 19:03:11.271601 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.271498    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"
Jul 26 19:03:11.271601 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.271537    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s" event={"ID":"05b08811-4f6e-496a-af71-e44bcc8d5887","Type":"ContainerDied","Data":"f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925"}
Jul 26 19:03:11.271601 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.271589    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s" event={"ID":"05b08811-4f6e-496a-af71-e44bcc8d5887","Type":"ContainerDied","Data":"49bf3aae4fd2f87148f86058371143899b183006ce0a04e992a25ee24277e74f"}
Jul 26 19:03:11.271770 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.271635    2559 scope.go:117] "RemoveContainer" containerID="f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925"
Jul 26 19:03:11.272960 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.272938    2559 generic.go:356] "Generic (PLEG): container finished" podID="ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d" containerID="f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71" exitCode=0
Jul 26 19:03:11.273068 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.272984    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"
Jul 26 19:03:11.273068 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.273025    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565" event={"ID":"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d","Type":"ContainerDied","Data":"f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71"}
Jul 26 19:03:11.273157 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.273078    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565" event={"ID":"ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d","Type":"ContainerDied","Data":"c2b256a821704764c8a8aaafd7822ea4a1799af6c6afa4dd759f806ed255b440"}
Jul 26 19:03:11.280540 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.280525    2559 scope.go:117] "RemoveContainer" containerID="f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925"
Jul 26 19:03:11.280818 ip-10-0-138-70 kubenswrapper[2559]: E0726 19:03:11.280801    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925\": container with ID starting with f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925 not found: ID does not exist" containerID="f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925"
Jul 26 19:03:11.280882 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.280827    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925"} err="failed to get container status \"f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925\": rpc error: code = NotFound desc = could not find container \"f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925\": container with ID starting with f6241be382620a6ab65f7bac5d439e3b656f9f6875663542d042a356e7017925 not found: ID does not exist"
Jul 26 19:03:11.280882 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.280844    2559 scope.go:117] "RemoveContainer" containerID="f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71"
Jul 26 19:03:11.288773 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.288750    2559 scope.go:117] "RemoveContainer" containerID="f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71"
Jul 26 19:03:11.289231 ip-10-0-138-70 kubenswrapper[2559]: E0726 19:03:11.289202    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71\": container with ID starting with f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71 not found: ID does not exist" containerID="f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71"
Jul 26 19:03:11.289304 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.289241    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71"} err="failed to get container status \"f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71\": rpc error: code = NotFound desc = could not find container \"f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71\": container with ID starting with f8ca78e2e6e9ba6e600668ac39c03328760c7d9e58e491f9e89c840b7d7fcd71 not found: ID does not exist"
Jul 26 19:03:11.290910 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.290890    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"]
Jul 26 19:03:11.296523 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.296502    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4735db-656f8b7d68-xh565"]
Jul 26 19:03:11.309575 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.309552    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"]
Jul 26 19:03:11.313627 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:11.313596    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4735db-9ddcdd6f6-vqx9s"]
Jul 26 19:03:13.178139 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:13.178105    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05b08811-4f6e-496a-af71-e44bcc8d5887" path="/var/lib/kubelet/pods/05b08811-4f6e-496a-af71-e44bcc8d5887/volumes"
Jul 26 19:03:13.178632 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:13.178524    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d" path="/var/lib/kubelet/pods/ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d/volumes"
Jul 26 19:03:20.234057 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:20.234025    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"]
Jul 26 19:03:35.253632 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.253591    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh" podUID="30228679-2f7e-47bc-80a1-94ed89c06ebf" containerName="main" containerID="cri-o://2cab069b0b9aafe996cb728eabd11d45a703b362423e0a95950494258ccdfd4b" gracePeriod=45
Jul 26 19:03:35.402200 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.402169    2559 generic.go:356] "Generic (PLEG): container finished" podID="30228679-2f7e-47bc-80a1-94ed89c06ebf" containerID="2cab069b0b9aafe996cb728eabd11d45a703b362423e0a95950494258ccdfd4b" exitCode=0
Jul 26 19:03:35.402360 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.402255    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh" event={"ID":"30228679-2f7e-47bc-80a1-94ed89c06ebf","Type":"ContainerDied","Data":"2cab069b0b9aafe996cb728eabd11d45a703b362423e0a95950494258ccdfd4b"}
Jul 26 19:03:35.510761 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.510705    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:03:35.594283 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594252    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-tmp-dir\") pod \"30228679-2f7e-47bc-80a1-94ed89c06ebf\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") "
Jul 26 19:03:35.594436 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594292    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-home\") pod \"30228679-2f7e-47bc-80a1-94ed89c06ebf\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") "
Jul 26 19:03:35.594436 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594316    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-dshm\") pod \"30228679-2f7e-47bc-80a1-94ed89c06ebf\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") "
Jul 26 19:03:35.594436 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594359    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-model-cache\") pod \"30228679-2f7e-47bc-80a1-94ed89c06ebf\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") "
Jul 26 19:03:35.594436 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594389    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30228679-2f7e-47bc-80a1-94ed89c06ebf-tls-certs\") pod \"30228679-2f7e-47bc-80a1-94ed89c06ebf\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") "
Jul 26 19:03:35.594675 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594542    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "30228679-2f7e-47bc-80a1-94ed89c06ebf" (UID: "30228679-2f7e-47bc-80a1-94ed89c06ebf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:03:35.594675 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594564    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jbxkh\" (UniqueName: \"kubernetes.io/projected/30228679-2f7e-47bc-80a1-94ed89c06ebf-kube-api-access-jbxkh\") pod \"30228679-2f7e-47bc-80a1-94ed89c06ebf\" (UID: \"30228679-2f7e-47bc-80a1-94ed89c06ebf\") "
Jul 26 19:03:35.594675 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594573    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-home" (OuterVolumeSpecName: "home") pod "30228679-2f7e-47bc-80a1-94ed89c06ebf" (UID: "30228679-2f7e-47bc-80a1-94ed89c06ebf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:03:35.594675 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594645    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-model-cache" (OuterVolumeSpecName: "model-cache") pod "30228679-2f7e-47bc-80a1-94ed89c06ebf" (UID: "30228679-2f7e-47bc-80a1-94ed89c06ebf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:03:35.594945 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594928    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-tmp-dir\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:35.595002 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594949    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-home\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:35.595002 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.594962    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-model-cache\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:35.596507 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.596482    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30228679-2f7e-47bc-80a1-94ed89c06ebf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "30228679-2f7e-47bc-80a1-94ed89c06ebf" (UID: "30228679-2f7e-47bc-80a1-94ed89c06ebf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 26 19:03:35.596637 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.596536    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30228679-2f7e-47bc-80a1-94ed89c06ebf-kube-api-access-jbxkh" (OuterVolumeSpecName: "kube-api-access-jbxkh") pod "30228679-2f7e-47bc-80a1-94ed89c06ebf" (UID: "30228679-2f7e-47bc-80a1-94ed89c06ebf"). InnerVolumeSpecName "kube-api-access-jbxkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:03:35.596637 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.596567    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-dshm" (OuterVolumeSpecName: "dshm") pod "30228679-2f7e-47bc-80a1-94ed89c06ebf" (UID: "30228679-2f7e-47bc-80a1-94ed89c06ebf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 26 19:03:35.696391 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.696366    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30228679-2f7e-47bc-80a1-94ed89c06ebf-tls-certs\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:35.696391 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.696387    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbxkh\" (UniqueName: \"kubernetes.io/projected/30228679-2f7e-47bc-80a1-94ed89c06ebf-kube-api-access-jbxkh\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:35.696391 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:35.696396    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30228679-2f7e-47bc-80a1-94ed89c06ebf-dshm\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:03:36.408323 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:36.408289    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"
Jul 26 19:03:36.408323 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:36.408318    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh" event={"ID":"30228679-2f7e-47bc-80a1-94ed89c06ebf","Type":"ContainerDied","Data":"a024b0ba37fb563bfa595784536a952acfe905510c9f880b4cdc075a8df974b8"}
Jul 26 19:03:36.408896 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:36.408374    2559 scope.go:117] "RemoveContainer" containerID="2cab069b0b9aafe996cb728eabd11d45a703b362423e0a95950494258ccdfd4b"
Jul 26 19:03:36.431963 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:36.431940    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"]
Jul 26 19:03:36.436888 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:36.436866    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-4735db/ipp-route-4735db-kserve-56cb5998c5-7jpxh"]
Jul 26 19:03:37.176576 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:03:37.176543    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30228679-2f7e-47bc-80a1-94ed89c06ebf" path="/var/lib/kubelet/pods/30228679-2f7e-47bc-80a1-94ed89c06ebf/volumes"
Jul 26 19:04:03.653630 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.653591    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"]
Jul 26 19:04:03.654095 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654076    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5269594f-9a54-42c4-8429-9761aa2b088f" containerName="revoke-keys"
Jul 26 19:04:03.654152 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654098    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5269594f-9a54-42c4-8429-9761aa2b088f" containerName="revoke-keys"
Jul 26 19:04:03.654152 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654118    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05b08811-4f6e-496a-af71-e44bcc8d5887" containerName="payload-processing"
Jul 26 19:04:03.654152 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654124    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="05b08811-4f6e-496a-af71-e44bcc8d5887" containerName="payload-processing"
Jul 26 19:04:03.654152 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654135    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30228679-2f7e-47bc-80a1-94ed89c06ebf" containerName="main"
Jul 26 19:04:03.654152 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654148    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="30228679-2f7e-47bc-80a1-94ed89c06ebf" containerName="main"
Jul 26 19:04:03.654326 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654158    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d" containerName="payload-pre-processing"
Jul 26 19:04:03.654326 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654164    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d" containerName="payload-pre-processing"
Jul 26 19:04:03.654326 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654172    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab1de98b-cd4a-4df9-b299-79c781a143a4" containerName="payload-pre-processing"
Jul 26 19:04:03.654326 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654178    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab1de98b-cd4a-4df9-b299-79c781a143a4" containerName="payload-pre-processing"
Jul 26 19:04:03.654326 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654237    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5269594f-9a54-42c4-8429-9761aa2b088f" containerName="revoke-keys"
Jul 26 19:04:03.654326 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654247    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="05b08811-4f6e-496a-af71-e44bcc8d5887" containerName="payload-processing"
Jul 26 19:04:03.654326 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654253    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab1de98b-cd4a-4df9-b299-79c781a143a4" containerName="payload-pre-processing"
Jul 26 19:04:03.654326 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654259    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebb24bfc-b62c-4bb8-bce5-6f9e9b5b3a7d" containerName="payload-pre-processing"
Jul 26 19:04:03.654326 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.654267    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="30228679-2f7e-47bc-80a1-94ed89c06ebf" containerName="main"
Jul 26 19:04:03.657463 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.657446    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"
Jul 26 19:04:03.660525 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.660502    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 19:04:03.660669 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.660532    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 19:04:03.665875 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.665849    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"]
Jul 26 19:04:03.748300 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.748261    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e196f85-279b-4c0b-9a8f-59fc6bcc2a96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29\" (UID: \"1e196f85-279b-4c0b-9a8f-59fc6bcc2a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"
Jul 26 19:04:03.848903 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.848870    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e196f85-279b-4c0b-9a8f-59fc6bcc2a96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29\" (UID: \"1e196f85-279b-4c0b-9a8f-59fc6bcc2a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"
Jul 26 19:04:03.849479 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.849460    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e196f85-279b-4c0b-9a8f-59fc6bcc2a96-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29\" (UID: \"1e196f85-279b-4c0b-9a8f-59fc6bcc2a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"
Jul 26 19:04:03.969310 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:03.969284    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"
Jul 26 19:04:04.104243 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:04.104215    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"]
Jul 26 19:04:04.107255 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:04:04.107223    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e196f85_279b_4c0b_9a8f_59fc6bcc2a96.slice/crio-fda7d3514bff2a7bcec1967b1a0beac0b0e7630b5e34cf31ad6d507b90f4083f WatchSource:0}: Error finding container fda7d3514bff2a7bcec1967b1a0beac0b0e7630b5e34cf31ad6d507b90f4083f: Status 404 returned error can't find the container with id fda7d3514bff2a7bcec1967b1a0beac0b0e7630b5e34cf31ad6d507b90f4083f
Jul 26 19:04:04.549242 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:04.549140    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29" event={"ID":"1e196f85-279b-4c0b-9a8f-59fc6bcc2a96","Type":"ContainerStarted","Data":"748d9bafd2fe6dc8a7c3aab83aae863806320a73b9f618bca27987536ef326e0"}
Jul 26 19:04:04.549242 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:04.549195    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29" event={"ID":"1e196f85-279b-4c0b-9a8f-59fc6bcc2a96","Type":"ContainerStarted","Data":"fda7d3514bff2a7bcec1967b1a0beac0b0e7630b5e34cf31ad6d507b90f4083f"}
Jul 26 19:04:04.568032 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:04.567983    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29" podStartSLOduration=1.567970008 podStartE2EDuration="1.567970008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:04:04.564700333 +0000 UTC m=+3115.889222429" watchObservedRunningTime="2026-07-26 19:04:04.567970008 +0000 UTC m=+3115.892492104"
Jul 26 19:04:05.563055 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:05.563021    2559 generic.go:356] "Generic (PLEG): container finished" podID="1e196f85-279b-4c0b-9a8f-59fc6bcc2a96" containerID="748d9bafd2fe6dc8a7c3aab83aae863806320a73b9f618bca27987536ef326e0" exitCode=0
Jul 26 19:04:05.563442 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:05.563105    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29" event={"ID":"1e196f85-279b-4c0b-9a8f-59fc6bcc2a96","Type":"ContainerDied","Data":"748d9bafd2fe6dc8a7c3aab83aae863806320a73b9f618bca27987536ef326e0"}
Jul 26 19:04:06.705229 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:06.705202    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"
Jul 26 19:04:06.777517 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:06.777487    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e196f85-279b-4c0b-9a8f-59fc6bcc2a96-maas-api-service-ca\") pod \"1e196f85-279b-4c0b-9a8f-59fc6bcc2a96\" (UID: \"1e196f85-279b-4c0b-9a8f-59fc6bcc2a96\") "
Jul 26 19:04:06.777841 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:06.777821    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e196f85-279b-4c0b-9a8f-59fc6bcc2a96-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e196f85-279b-4c0b-9a8f-59fc6bcc2a96" (UID: "1e196f85-279b-4c0b-9a8f-59fc6bcc2a96"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:06.878826 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:06.878752    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e196f85-279b-4c0b-9a8f-59fc6bcc2a96-maas-api-service-ca\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:07.573863 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:07.573781    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29" event={"ID":"1e196f85-279b-4c0b-9a8f-59fc6bcc2a96","Type":"ContainerDied","Data":"fda7d3514bff2a7bcec1967b1a0beac0b0e7630b5e34cf31ad6d507b90f4083f"}
Jul 26 19:04:07.573863 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:07.573805    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"
Jul 26 19:04:07.573863 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:07.573834    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fda7d3514bff2a7bcec1967b1a0beac0b0e7630b5e34cf31ad6d507b90f4083f"
Jul 26 19:04:14.154159 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.154116    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"]
Jul 26 19:04:14.154632 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.154341    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr" podUID="79f61a20-c303-4325-848c-e3c9764701eb" containerName="payload-processing" containerID="cri-o://aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730" gracePeriod=30
Jul 26 19:04:14.162876 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.162848    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"]
Jul 26 19:04:14.163095 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.163072    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv" podUID="1c274fd7-320f-4ad2-b792-f1f38fa9128a" containerName="payload-pre-processing" containerID="cri-o://4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648" gracePeriod=30
Jul 26 19:04:14.440053 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.440021    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:04:14.442937 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.442913    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:04:14.546963 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.546936    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfpp7\" (UniqueName: \"kubernetes.io/projected/79f61a20-c303-4325-848c-e3c9764701eb-kube-api-access-bfpp7\") pod \"79f61a20-c303-4325-848c-e3c9764701eb\" (UID: \"79f61a20-c303-4325-848c-e3c9764701eb\") "
Jul 26 19:04:14.547119 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.546976    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2sj64\" (UniqueName: \"kubernetes.io/projected/1c274fd7-320f-4ad2-b792-f1f38fa9128a-kube-api-access-2sj64\") pod \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\" (UID: \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\") "
Jul 26 19:04:14.547119 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.547021    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79f61a20-c303-4325-848c-e3c9764701eb-plugins-config-volume\") pod \"79f61a20-c303-4325-848c-e3c9764701eb\" (UID: \"79f61a20-c303-4325-848c-e3c9764701eb\") "
Jul 26 19:04:14.547207 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.547185    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c274fd7-320f-4ad2-b792-f1f38fa9128a-plugins-config-volume\") pod \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\" (UID: \"1c274fd7-320f-4ad2-b792-f1f38fa9128a\") "
Jul 26 19:04:14.547425 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.547400    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79f61a20-c303-4325-848c-e3c9764701eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79f61a20-c303-4325-848c-e3c9764701eb" (UID: "79f61a20-c303-4325-848c-e3c9764701eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:14.547573 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.547514    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79f61a20-c303-4325-848c-e3c9764701eb-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:14.547573 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.547553    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c274fd7-320f-4ad2-b792-f1f38fa9128a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1c274fd7-320f-4ad2-b792-f1f38fa9128a" (UID: "1c274fd7-320f-4ad2-b792-f1f38fa9128a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:14.549264 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.549244    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c274fd7-320f-4ad2-b792-f1f38fa9128a-kube-api-access-2sj64" (OuterVolumeSpecName: "kube-api-access-2sj64") pod "1c274fd7-320f-4ad2-b792-f1f38fa9128a" (UID: "1c274fd7-320f-4ad2-b792-f1f38fa9128a"). InnerVolumeSpecName "kube-api-access-2sj64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:14.549347 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.549271    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f61a20-c303-4325-848c-e3c9764701eb-kube-api-access-bfpp7" (OuterVolumeSpecName: "kube-api-access-bfpp7") pod "79f61a20-c303-4325-848c-e3c9764701eb" (UID: "79f61a20-c303-4325-848c-e3c9764701eb"). InnerVolumeSpecName "kube-api-access-bfpp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:14.608115 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.608086    2559 generic.go:356] "Generic (PLEG): container finished" podID="1c274fd7-320f-4ad2-b792-f1f38fa9128a" containerID="4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648" exitCode=0
Jul 26 19:04:14.608234 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.608166    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"
Jul 26 19:04:14.608234 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.608199    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv" event={"ID":"1c274fd7-320f-4ad2-b792-f1f38fa9128a","Type":"ContainerDied","Data":"4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648"}
Jul 26 19:04:14.608365 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.608251    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv" event={"ID":"1c274fd7-320f-4ad2-b792-f1f38fa9128a","Type":"ContainerDied","Data":"442ebe066c9bfc535eaecf30c3a65d8cb38ac9fe73f080cbbca847acaa70faf4"}
Jul 26 19:04:14.608365 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.608284    2559 scope.go:117] "RemoveContainer" containerID="4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648"
Jul 26 19:04:14.609689 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.609662    2559 generic.go:356] "Generic (PLEG): container finished" podID="79f61a20-c303-4325-848c-e3c9764701eb" containerID="aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730" exitCode=0
Jul 26 19:04:14.609796 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.609720    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr" event={"ID":"79f61a20-c303-4325-848c-e3c9764701eb","Type":"ContainerDied","Data":"aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730"}
Jul 26 19:04:14.609796 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.609726    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"
Jul 26 19:04:14.609796 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.609784    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr" event={"ID":"79f61a20-c303-4325-848c-e3c9764701eb","Type":"ContainerDied","Data":"bda637f2f08d38150f23debbb4879b3afe15946038d1ad6ec89eeff04db0f7b0"}
Jul 26 19:04:14.618221 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.618199    2559 scope.go:117] "RemoveContainer" containerID="4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648"
Jul 26 19:04:14.618528 ip-10-0-138-70 kubenswrapper[2559]: E0726 19:04:14.618505    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648\": container with ID starting with 4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648 not found: ID does not exist" containerID="4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648"
Jul 26 19:04:14.618656 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.618538    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648"} err="failed to get container status \"4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648\": rpc error: code = NotFound desc = could not find container \"4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648\": container with ID starting with 4834ea262038fb306437c443c4f06182e699379cf52a5446b6aa3c514e729648 not found: ID does not exist"
Jul 26 19:04:14.618656 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.618562    2559 scope.go:117] "RemoveContainer" containerID="aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730"
Jul 26 19:04:14.626411 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.626393    2559 scope.go:117] "RemoveContainer" containerID="aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730"
Jul 26 19:04:14.626665 ip-10-0-138-70 kubenswrapper[2559]: E0726 19:04:14.626650    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730\": container with ID starting with aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730 not found: ID does not exist" containerID="aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730"
Jul 26 19:04:14.626722 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.626680    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730"} err="failed to get container status \"aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730\": rpc error: code = NotFound desc = could not find container \"aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730\": container with ID starting with aa56dd8c2adc04449e8700c296067b298e79870bf6080b4dcd0a868f204e1730 not found: ID does not exist"
Jul 26 19:04:14.634810 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.634789    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"]
Jul 26 19:04:14.639187 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.639161    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1318a7-649977c77d-nvfzr"]
Jul 26 19:04:14.648012 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.647992    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c274fd7-320f-4ad2-b792-f1f38fa9128a-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:14.648091 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.648013    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfpp7\" (UniqueName: \"kubernetes.io/projected/79f61a20-c303-4325-848c-e3c9764701eb-kube-api-access-bfpp7\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:14.648091 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.648024    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sj64\" (UniqueName: \"kubernetes.io/projected/1c274fd7-320f-4ad2-b792-f1f38fa9128a-kube-api-access-2sj64\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:14.650810 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.650790    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"]
Jul 26 19:04:14.657056 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:14.657031    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-1318a7-8556d7b89b-7b8sv"]
Jul 26 19:04:15.182760 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:15.182728    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c274fd7-320f-4ad2-b792-f1f38fa9128a" path="/var/lib/kubelet/pods/1c274fd7-320f-4ad2-b792-f1f38fa9128a/volumes"
Jul 26 19:04:15.183125 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:15.183080    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79f61a20-c303-4325-848c-e3c9764701eb" path="/var/lib/kubelet/pods/79f61a20-c303-4325-848c-e3c9764701eb/volumes"
Jul 26 19:04:18.737256 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:18.737224    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"]
Jul 26 19:04:18.741044 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:18.741016    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-1318a7-8nv29"]
Jul 26 19:04:19.178447 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:19.178416    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e196f85-279b-4c0b-9a8f-59fc6bcc2a96" path="/var/lib/kubelet/pods/1e196f85-279b-4c0b-9a8f-59fc6bcc2a96/volumes"
Jul 26 19:04:25.642599 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.642558    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"]
Jul 26 19:04:25.643261 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.643239    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e196f85-279b-4c0b-9a8f-59fc6bcc2a96" containerName="revoke-keys"
Jul 26 19:04:25.643304 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.643265    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e196f85-279b-4c0b-9a8f-59fc6bcc2a96" containerName="revoke-keys"
Jul 26 19:04:25.643304 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.643282    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f61a20-c303-4325-848c-e3c9764701eb" containerName="payload-processing"
Jul 26 19:04:25.643304 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.643291    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f61a20-c303-4325-848c-e3c9764701eb" containerName="payload-processing"
Jul 26 19:04:25.643402 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.643316    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c274fd7-320f-4ad2-b792-f1f38fa9128a" containerName="payload-pre-processing"
Jul 26 19:04:25.643402 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.643326    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c274fd7-320f-4ad2-b792-f1f38fa9128a" containerName="payload-pre-processing"
Jul 26 19:04:25.643470 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.643424    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e196f85-279b-4c0b-9a8f-59fc6bcc2a96" containerName="revoke-keys"
Jul 26 19:04:25.643470 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.643435    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f61a20-c303-4325-848c-e3c9764701eb" containerName="payload-processing"
Jul 26 19:04:25.643470 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.643447    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c274fd7-320f-4ad2-b792-f1f38fa9128a" containerName="payload-pre-processing"
Jul 26 19:04:25.646679 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.646661    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"
Jul 26 19:04:25.649948 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.649917    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 19:04:25.650088 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.649951    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 19:04:25.654665 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.654407    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"]
Jul 26 19:04:25.751389 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.751349    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95e7f757-a689-4346-bbd3-432620458e24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb\" (UID: \"95e7f757-a689-4346-bbd3-432620458e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"
Jul 26 19:04:25.852715 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.852687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95e7f757-a689-4346-bbd3-432620458e24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb\" (UID: \"95e7f757-a689-4346-bbd3-432620458e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"
Jul 26 19:04:25.853302 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.853279    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95e7f757-a689-4346-bbd3-432620458e24-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb\" (UID: \"95e7f757-a689-4346-bbd3-432620458e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"
Jul 26 19:04:25.958974 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:25.958947    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"
Jul 26 19:04:26.100853 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:26.100767    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"]
Jul 26 19:04:26.103186 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:04:26.103157    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e7f757_a689_4346_bbd3_432620458e24.slice/crio-4ee67ae01cb2c30eb2497f815798ac346f879b0e7df3025216471ed350e26b49 WatchSource:0}: Error finding container 4ee67ae01cb2c30eb2497f815798ac346f879b0e7df3025216471ed350e26b49: Status 404 returned error can't find the container with id 4ee67ae01cb2c30eb2497f815798ac346f879b0e7df3025216471ed350e26b49
Jul 26 19:04:26.674139 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:26.674101    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb" event={"ID":"95e7f757-a689-4346-bbd3-432620458e24","Type":"ContainerStarted","Data":"f6a13d4bf97d0e5d3e3b7fdd3ccc4b034188b5bfc509c512860eb69c5edf91b9"}
Jul 26 19:04:26.674568 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:26.674158    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb" event={"ID":"95e7f757-a689-4346-bbd3-432620458e24","Type":"ContainerStarted","Data":"4ee67ae01cb2c30eb2497f815798ac346f879b0e7df3025216471ed350e26b49"}
Jul 26 19:04:26.692085 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:26.692037    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb" podStartSLOduration=1.6920212430000001 podStartE2EDuration="1.692021243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:04:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:04:26.689698032 +0000 UTC m=+3138.014220131" watchObservedRunningTime="2026-07-26 19:04:26.692021243 +0000 UTC m=+3138.016543342"
Jul 26 19:04:27.680594 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:27.680559    2559 generic.go:356] "Generic (PLEG): container finished" podID="95e7f757-a689-4346-bbd3-432620458e24" containerID="f6a13d4bf97d0e5d3e3b7fdd3ccc4b034188b5bfc509c512860eb69c5edf91b9" exitCode=0
Jul 26 19:04:27.680999 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:27.680652    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb" event={"ID":"95e7f757-a689-4346-bbd3-432620458e24","Type":"ContainerDied","Data":"f6a13d4bf97d0e5d3e3b7fdd3ccc4b034188b5bfc509c512860eb69c5edf91b9"}
Jul 26 19:04:28.819110 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:28.819081    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"
Jul 26 19:04:28.983731 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:28.983648    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95e7f757-a689-4346-bbd3-432620458e24-maas-api-service-ca\") pod \"95e7f757-a689-4346-bbd3-432620458e24\" (UID: \"95e7f757-a689-4346-bbd3-432620458e24\") "
Jul 26 19:04:28.984009 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:28.983986    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95e7f757-a689-4346-bbd3-432620458e24-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95e7f757-a689-4346-bbd3-432620458e24" (UID: "95e7f757-a689-4346-bbd3-432620458e24"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:29.084387 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:29.084355    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95e7f757-a689-4346-bbd3-432620458e24-maas-api-service-ca\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:29.691483 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:29.691454    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"
Jul 26 19:04:29.691483 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:29.691465    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb" event={"ID":"95e7f757-a689-4346-bbd3-432620458e24","Type":"ContainerDied","Data":"4ee67ae01cb2c30eb2497f815798ac346f879b0e7df3025216471ed350e26b49"}
Jul 26 19:04:29.691719 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:29.691513    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ee67ae01cb2c30eb2497f815798ac346f879b0e7df3025216471ed350e26b49"
Jul 26 19:04:36.152745 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.152662    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"]
Jul 26 19:04:36.153177 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.152910    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b" podUID="622ef703-a7d2-4556-adb3-45e2783901a8" containerName="payload-processing" containerID="cri-o://48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00" gracePeriod=30
Jul 26 19:04:36.173686 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.173649    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"]
Jul 26 19:04:36.173948 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.173901    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn" podUID="e12efcc8-f2ed-4ac3-aa3f-7764675d315c" containerName="payload-pre-processing" containerID="cri-o://41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476" gracePeriod=30
Jul 26 19:04:36.431901 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.431877    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 19:04:36.441808 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.441778    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/622ef703-a7d2-4556-adb3-45e2783901a8-plugins-config-volume\") pod \"622ef703-a7d2-4556-adb3-45e2783901a8\" (UID: \"622ef703-a7d2-4556-adb3-45e2783901a8\") "
Jul 26 19:04:36.441955 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.441917    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7hgd\" (UniqueName: \"kubernetes.io/projected/622ef703-a7d2-4556-adb3-45e2783901a8-kube-api-access-l7hgd\") pod \"622ef703-a7d2-4556-adb3-45e2783901a8\" (UID: \"622ef703-a7d2-4556-adb3-45e2783901a8\") "
Jul 26 19:04:36.442174 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.442143    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/622ef703-a7d2-4556-adb3-45e2783901a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "622ef703-a7d2-4556-adb3-45e2783901a8" (UID: "622ef703-a7d2-4556-adb3-45e2783901a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:36.444393 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.444372    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/622ef703-a7d2-4556-adb3-45e2783901a8-kube-api-access-l7hgd" (OuterVolumeSpecName: "kube-api-access-l7hgd") pod "622ef703-a7d2-4556-adb3-45e2783901a8" (UID: "622ef703-a7d2-4556-adb3-45e2783901a8"). InnerVolumeSpecName "kube-api-access-l7hgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:36.461037 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.461008    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 19:04:36.543276 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.543248    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4pk4\" (UniqueName: \"kubernetes.io/projected/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-kube-api-access-c4pk4\") pod \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\" (UID: \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\") "
Jul 26 19:04:36.543419 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.543317    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-plugins-config-volume\") pod \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\" (UID: \"e12efcc8-f2ed-4ac3-aa3f-7764675d315c\") "
Jul 26 19:04:36.543600 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.543585    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7hgd\" (UniqueName: \"kubernetes.io/projected/622ef703-a7d2-4556-adb3-45e2783901a8-kube-api-access-l7hgd\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:36.543671 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.543605    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/622ef703-a7d2-4556-adb3-45e2783901a8-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:36.543748 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.543726    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e12efcc8-f2ed-4ac3-aa3f-7764675d315c" (UID: "e12efcc8-f2ed-4ac3-aa3f-7764675d315c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:36.545300 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.545282    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-kube-api-access-c4pk4" (OuterVolumeSpecName: "kube-api-access-c4pk4") pod "e12efcc8-f2ed-4ac3-aa3f-7764675d315c" (UID: "e12efcc8-f2ed-4ac3-aa3f-7764675d315c"). InnerVolumeSpecName "kube-api-access-c4pk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:36.644673 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.644641    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:36.644673 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.644673    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4pk4\" (UniqueName: \"kubernetes.io/projected/e12efcc8-f2ed-4ac3-aa3f-7764675d315c-kube-api-access-c4pk4\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:36.726738 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.726707    2559 generic.go:356] "Generic (PLEG): container finished" podID="e12efcc8-f2ed-4ac3-aa3f-7764675d315c" containerID="41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476" exitCode=0
Jul 26 19:04:36.726893 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.726757    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"
Jul 26 19:04:36.726893 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.726792    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn" event={"ID":"e12efcc8-f2ed-4ac3-aa3f-7764675d315c","Type":"ContainerDied","Data":"41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476"}
Jul 26 19:04:36.726893 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.726849    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn" event={"ID":"e12efcc8-f2ed-4ac3-aa3f-7764675d315c","Type":"ContainerDied","Data":"f2d14b61ca1ce9a77c4792c9840e8aad8f46aa47b04700b05adbcd21f6e11b69"}
Jul 26 19:04:36.726893 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.726879    2559 scope.go:117] "RemoveContainer" containerID="41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476"
Jul 26 19:04:36.728162 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.728141    2559 generic.go:356] "Generic (PLEG): container finished" podID="622ef703-a7d2-4556-adb3-45e2783901a8" containerID="48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00" exitCode=0
Jul 26 19:04:36.728249 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.728172    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"
Jul 26 19:04:36.728249 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.728185    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b" event={"ID":"622ef703-a7d2-4556-adb3-45e2783901a8","Type":"ContainerDied","Data":"48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00"}
Jul 26 19:04:36.728249 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.728234    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b" event={"ID":"622ef703-a7d2-4556-adb3-45e2783901a8","Type":"ContainerDied","Data":"cbf5e4ec4429d75da9dd551fa02c16031a7bfccc84e2ca58a7d05fecba0559a8"}
Jul 26 19:04:36.736505 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.736488    2559 scope.go:117] "RemoveContainer" containerID="41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476"
Jul 26 19:04:36.736809 ip-10-0-138-70 kubenswrapper[2559]: E0726 19:04:36.736789    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476\": container with ID starting with 41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476 not found: ID does not exist" containerID="41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476"
Jul 26 19:04:36.736897 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.736815    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476"} err="failed to get container status \"41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476\": rpc error: code = NotFound desc = could not find container \"41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476\": container with ID starting with 41460f52fce3cdeb9cb088a4ca4538d20622b64aae9cd45145bb2407c8982476 not found: ID does not exist"
Jul 26 19:04:36.736897 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.736832    2559 scope.go:117] "RemoveContainer" containerID="48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00"
Jul 26 19:04:36.744895 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.744875    2559 scope.go:117] "RemoveContainer" containerID="48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00"
Jul 26 19:04:36.745162 ip-10-0-138-70 kubenswrapper[2559]: E0726 19:04:36.745141    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00\": container with ID starting with 48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00 not found: ID does not exist" containerID="48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00"
Jul 26 19:04:36.745231 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.745169    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00"} err="failed to get container status \"48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00\": rpc error: code = NotFound desc = could not find container \"48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00\": container with ID starting with 48ace381c954a0e14f048929e6efbe4cb97231892102cbf16f54f6a497344c00 not found: ID does not exist"
Jul 26 19:04:36.764566 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.764542    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"]
Jul 26 19:04:36.774889 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.774869    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-2b193e-7859f89586-rjz5b"]
Jul 26 19:04:36.809183 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.809158    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"]
Jul 26 19:04:36.816097 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:36.816077    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-2b193e-58f954bc9b-98wmn"]
Jul 26 19:04:37.178143 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:37.178108    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="622ef703-a7d2-4556-adb3-45e2783901a8" path="/var/lib/kubelet/pods/622ef703-a7d2-4556-adb3-45e2783901a8/volumes"
Jul 26 19:04:37.178480 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:37.178467    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e12efcc8-f2ed-4ac3-aa3f-7764675d315c" path="/var/lib/kubelet/pods/e12efcc8-f2ed-4ac3-aa3f-7764675d315c/volumes"
Jul 26 19:04:40.728657 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:40.728610    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"]
Jul 26 19:04:40.731290 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:40.731267    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-2b193e-pptjb"]
Jul 26 19:04:41.177504 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:41.177446    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95e7f757-a689-4346-bbd3-432620458e24" path="/var/lib/kubelet/pods/95e7f757-a689-4346-bbd3-432620458e24/volumes"
Jul 26 19:04:48.040083 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040051    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"]
Jul 26 19:04:48.040476 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040429    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95e7f757-a689-4346-bbd3-432620458e24" containerName="revoke-keys"
Jul 26 19:04:48.040476 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040440    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="95e7f757-a689-4346-bbd3-432620458e24" containerName="revoke-keys"
Jul 26 19:04:48.040476 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040453    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="622ef703-a7d2-4556-adb3-45e2783901a8" containerName="payload-processing"
Jul 26 19:04:48.040476 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040459    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="622ef703-a7d2-4556-adb3-45e2783901a8" containerName="payload-processing"
Jul 26 19:04:48.040476 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040474    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e12efcc8-f2ed-4ac3-aa3f-7764675d315c" containerName="payload-pre-processing"
Jul 26 19:04:48.040708 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040481    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e12efcc8-f2ed-4ac3-aa3f-7764675d315c" containerName="payload-pre-processing"
Jul 26 19:04:48.040708 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040638    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="95e7f757-a689-4346-bbd3-432620458e24" containerName="revoke-keys"
Jul 26 19:04:48.040708 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040654    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e12efcc8-f2ed-4ac3-aa3f-7764675d315c" containerName="payload-pre-processing"
Jul 26 19:04:48.040708 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.040680    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="622ef703-a7d2-4556-adb3-45e2783901a8" containerName="payload-processing"
Jul 26 19:04:48.045677 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.045651    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"
Jul 26 19:04:48.048707 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.048682    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 26 19:04:48.048898 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.048758    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wxnsx\""
Jul 26 19:04:48.052907 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.052871    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"]
Jul 26 19:04:48.157003 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.156965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c\" (UID: \"3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"
Jul 26 19:04:48.258724 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.258684    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c\" (UID: \"3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"
Jul 26 19:04:48.259297 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.259278    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c\" (UID: \"3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"
Jul 26 19:04:48.359244 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.359159    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"
Jul 26 19:04:48.496525 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.496493    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"]
Jul 26 19:04:48.497845 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:04:48.497816    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc7af8d_ceb7_4d15_980e_8d040ee7ca6d.slice/crio-ff88efb7e8ec6f0f2b2a8b92c5913928512bdc181a0da4e42ef99c2d2166ac55 WatchSource:0}: Error finding container ff88efb7e8ec6f0f2b2a8b92c5913928512bdc181a0da4e42ef99c2d2166ac55: Status 404 returned error can't find the container with id ff88efb7e8ec6f0f2b2a8b92c5913928512bdc181a0da4e42ef99c2d2166ac55
Jul 26 19:04:48.791234 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:48.791175    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c" event={"ID":"3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d","Type":"ContainerStarted","Data":"ff88efb7e8ec6f0f2b2a8b92c5913928512bdc181a0da4e42ef99c2d2166ac55"}
Jul 26 19:04:49.798092 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:49.798057    2559 generic.go:356] "Generic (PLEG): container finished" podID="3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d" containerID="a34f64db8ad02c0dc2c2b58a1457f5f9655eb8ddba64b56831391c55217bf2a6" exitCode=0
Jul 26 19:04:49.798495 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:49.798155    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c" event={"ID":"3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d","Type":"ContainerDied","Data":"a34f64db8ad02c0dc2c2b58a1457f5f9655eb8ddba64b56831391c55217bf2a6"}
Jul 26 19:04:50.936694 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:50.936647    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"
Jul 26 19:04:50.984787 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:50.984759    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d-maas-api-service-ca\") pod \"3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d\" (UID: \"3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d\") "
Jul 26 19:04:50.985114 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:50.985090    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d" (UID: "3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:51.086243 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:51.086220    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d-maas-api-service-ca\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:51.809505 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:51.809476    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"
Jul 26 19:04:51.809505 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:51.809493    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c" event={"ID":"3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d","Type":"ContainerDied","Data":"ff88efb7e8ec6f0f2b2a8b92c5913928512bdc181a0da4e42ef99c2d2166ac55"}
Jul 26 19:04:51.809738 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:51.809542    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff88efb7e8ec6f0f2b2a8b92c5913928512bdc181a0da4e42ef99c2d2166ac55"
Jul 26 19:04:58.572407 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.572364    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"]
Jul 26 19:04:58.572981 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.572743    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh" podUID="f86e051f-2f89-47d3-8880-acad32a4d2a3" containerName="payload-processing" containerID="cri-o://e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f" gracePeriod=30
Jul 26 19:04:58.582433 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.582098    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"]
Jul 26 19:04:58.582433 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.582315    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw" podUID="43fd86c7-c336-45f5-8952-a277646997fc" containerName="payload-pre-processing" containerID="cri-o://a8bee1a8afda8d98d776eb8103794024afe9b6cc260511b4efe4b44522e82bbf" gracePeriod=30
Jul 26 19:04:58.828406 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.828344    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 19:04:58.847526 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.847498    2559 generic.go:356] "Generic (PLEG): container finished" podID="43fd86c7-c336-45f5-8952-a277646997fc" containerID="a8bee1a8afda8d98d776eb8103794024afe9b6cc260511b4efe4b44522e82bbf" exitCode=0
Jul 26 19:04:58.847677 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.847574    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw" event={"ID":"43fd86c7-c336-45f5-8952-a277646997fc","Type":"ContainerDied","Data":"a8bee1a8afda8d98d776eb8103794024afe9b6cc260511b4efe4b44522e82bbf"}
Jul 26 19:04:58.847756 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.847685    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw" event={"ID":"43fd86c7-c336-45f5-8952-a277646997fc","Type":"ContainerDied","Data":"23878021b64effd1974028d26e61ea1f3618c009ff84aa3ebd4bd448c803eca7"}
Jul 26 19:04:58.847872 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.847846    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23878021b64effd1974028d26e61ea1f3618c009ff84aa3ebd4bd448c803eca7"
Jul 26 19:04:58.849547 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.849520    2559 generic.go:356] "Generic (PLEG): container finished" podID="f86e051f-2f89-47d3-8880-acad32a4d2a3" containerID="e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f" exitCode=0
Jul 26 19:04:58.849670 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.849578    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh" event={"ID":"f86e051f-2f89-47d3-8880-acad32a4d2a3","Type":"ContainerDied","Data":"e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f"}
Jul 26 19:04:58.849670 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.849637    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh" event={"ID":"f86e051f-2f89-47d3-8880-acad32a4d2a3","Type":"ContainerDied","Data":"bcfa23429943850e6fa0b1e887d637dc3a43fba3598dcdf307f5180a2514360c"}
Jul 26 19:04:58.849760 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.849672    2559 scope.go:117] "RemoveContainer" containerID="e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f"
Jul 26 19:04:58.849839 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.849826    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"
Jul 26 19:04:58.856209 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.856190    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 19:04:58.860560 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.860540    2559 scope.go:117] "RemoveContainer" containerID="e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f"
Jul 26 19:04:58.860846 ip-10-0-138-70 kubenswrapper[2559]: E0726 19:04:58.860823    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f\": container with ID starting with e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f not found: ID does not exist" containerID="e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f"
Jul 26 19:04:58.860909 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.860854    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f"} err="failed to get container status \"e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f\": rpc error: code = NotFound desc = could not find container \"e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f\": container with ID starting with e3d21c26e33b7d06e8e2fbb4d949fde0210a734821f9aee6a78a92dfdcdcda3f not found: ID does not exist"
Jul 26 19:04:58.957947 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.957919    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwqsz\" (UniqueName: \"kubernetes.io/projected/f86e051f-2f89-47d3-8880-acad32a4d2a3-kube-api-access-rwqsz\") pod \"f86e051f-2f89-47d3-8880-acad32a4d2a3\" (UID: \"f86e051f-2f89-47d3-8880-acad32a4d2a3\") "
Jul 26 19:04:58.958119 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.957981    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86e051f-2f89-47d3-8880-acad32a4d2a3-plugins-config-volume\") pod \"f86e051f-2f89-47d3-8880-acad32a4d2a3\" (UID: \"f86e051f-2f89-47d3-8880-acad32a4d2a3\") "
Jul 26 19:04:58.958119 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.958070    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-784vc\" (UniqueName: \"kubernetes.io/projected/43fd86c7-c336-45f5-8952-a277646997fc-kube-api-access-784vc\") pod \"43fd86c7-c336-45f5-8952-a277646997fc\" (UID: \"43fd86c7-c336-45f5-8952-a277646997fc\") "
Jul 26 19:04:58.958119 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.958111    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43fd86c7-c336-45f5-8952-a277646997fc-plugins-config-volume\") pod \"43fd86c7-c336-45f5-8952-a277646997fc\" (UID: \"43fd86c7-c336-45f5-8952-a277646997fc\") "
Jul 26 19:04:58.958468 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.958433    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f86e051f-2f89-47d3-8880-acad32a4d2a3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f86e051f-2f89-47d3-8880-acad32a4d2a3" (UID: "f86e051f-2f89-47d3-8880-acad32a4d2a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:58.958530 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.958476    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43fd86c7-c336-45f5-8952-a277646997fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "43fd86c7-c336-45f5-8952-a277646997fc" (UID: "43fd86c7-c336-45f5-8952-a277646997fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 26 19:04:58.960098 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.960078    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f86e051f-2f89-47d3-8880-acad32a4d2a3-kube-api-access-rwqsz" (OuterVolumeSpecName: "kube-api-access-rwqsz") pod "f86e051f-2f89-47d3-8880-acad32a4d2a3" (UID: "f86e051f-2f89-47d3-8880-acad32a4d2a3"). InnerVolumeSpecName "kube-api-access-rwqsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:58.960159 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:58.960084    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43fd86c7-c336-45f5-8952-a277646997fc-kube-api-access-784vc" (OuterVolumeSpecName: "kube-api-access-784vc") pod "43fd86c7-c336-45f5-8952-a277646997fc" (UID: "43fd86c7-c336-45f5-8952-a277646997fc"). InnerVolumeSpecName "kube-api-access-784vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 26 19:04:59.059241 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:59.059208    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86e051f-2f89-47d3-8880-acad32a4d2a3-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:59.059241 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:59.059234    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-784vc\" (UniqueName: \"kubernetes.io/projected/43fd86c7-c336-45f5-8952-a277646997fc-kube-api-access-784vc\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:59.059241 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:59.059243    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/43fd86c7-c336-45f5-8952-a277646997fc-plugins-config-volume\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:59.059450 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:59.059262    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwqsz\" (UniqueName: \"kubernetes.io/projected/f86e051f-2f89-47d3-8880-acad32a4d2a3-kube-api-access-rwqsz\") on node \"ip-10-0-138-70.ec2.internal\" DevicePath \"\""
Jul 26 19:04:59.178530 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:59.178504    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"]
Jul 26 19:04:59.178660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:59.178537    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-c75d5a-584d4965d6-h28xh"]
Jul 26 19:04:59.855271 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:59.855183    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"
Jul 26 19:04:59.873297 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:59.873264    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"]
Jul 26 19:04:59.877339 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:04:59.877317    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-c75d5a-5f7d57cf5b-lflsw"]
Jul 26 19:05:01.176919 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:01.176884    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43fd86c7-c336-45f5-8952-a277646997fc" path="/var/lib/kubelet/pods/43fd86c7-c336-45f5-8952-a277646997fc/volumes"
Jul 26 19:05:01.177290 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:01.177246    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f86e051f-2f89-47d3-8880-acad32a4d2a3" path="/var/lib/kubelet/pods/f86e051f-2f89-47d3-8880-acad32a4d2a3/volumes"
Jul 26 19:05:03.129793 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:03.129758    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"]
Jul 26 19:05:03.133915 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:03.133887    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-c75d5a-x4q6c"]
Jul 26 19:05:03.178492 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:03.178458    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d" path="/var/lib/kubelet/pods/3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d/volumes"
Jul 26 19:05:09.553969 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:09.553940    2559 scope.go:117] "RemoveContainer" containerID="a8bee1a8afda8d98d776eb8103794024afe9b6cc260511b4efe4b44522e82bbf"
Jul 26 19:05:15.888692 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:15.888659    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8494c84d99-k6zhp_402e9e41-e678-4722-b6a2-13a0af69febe/manager/0.log"
Jul 26 19:05:16.455466 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:16.455442    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-mqvph_05604bb7-666e-4786-a350-bea356d5e5a5/copy-manifests/0.log"
Jul 26 19:05:16.463441 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:16.463417    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-mqvph_05604bb7-666e-4786-a350-bea356d5e5a5/manager/0.log"
Jul 26 19:05:18.838921 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:18.838893    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-spb9v_504040e5-5cb4-4fc9-a037-ed70147096b7/kuadrant-console-plugin/0.log"
Jul 26 19:05:19.067032 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:19.066998    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-snpqf_a3606262-f9fa-4397-9d5c-dd4521f2c146/manager/0.log"
Jul 26 19:05:19.840670 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:19.840640    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c7c67d4f6-b8krv_89307c3c-e972-476b-963c-4df2e54d2f92/kube-auth-proxy/0.log"
Jul 26 19:05:20.736536 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:20.736502    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-94bl9_80f6993d-e8bf-424d-a64e-72b33c31c5e1/main/0.log"
Jul 26 19:05:20.848068 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:20.848036    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-f5xrl_50c5641b-ad19-4b2f-811a-15235db96773/main/0.log"
Jul 26 19:05:21.169379 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:21.169352    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ch4nr_93d6aaab-abbf-41af-bce4-91869932c551/main/0.log"
Jul 26 19:05:26.863037 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863000    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fjlfd/must-gather-mmb5b"]
Jul 26 19:05:26.863419 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863388    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f86e051f-2f89-47d3-8880-acad32a4d2a3" containerName="payload-processing"
Jul 26 19:05:26.863419 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863399    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="f86e051f-2f89-47d3-8880-acad32a4d2a3" containerName="payload-processing"
Jul 26 19:05:26.863419 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863408    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d" containerName="revoke-keys"
Jul 26 19:05:26.863419 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863414    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d" containerName="revoke-keys"
Jul 26 19:05:26.863555 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863426    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43fd86c7-c336-45f5-8952-a277646997fc" containerName="payload-pre-processing"
Jul 26 19:05:26.863555 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863432    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="43fd86c7-c336-45f5-8952-a277646997fc" containerName="payload-pre-processing"
Jul 26 19:05:26.863555 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863490    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dc7af8d-ceb7-4d15-980e-8d040ee7ca6d" containerName="revoke-keys"
Jul 26 19:05:26.863555 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863502    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="43fd86c7-c336-45f5-8952-a277646997fc" containerName="payload-pre-processing"
Jul 26 19:05:26.863555 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.863509    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="f86e051f-2f89-47d3-8880-acad32a4d2a3" containerName="payload-processing"
Jul 26 19:05:26.866351 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.866336    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fjlfd/must-gather-mmb5b"
Jul 26 19:05:26.869407 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.869386    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fjlfd\"/\"kube-root-ca.crt\""
Jul 26 19:05:26.869583 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.869565    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fjlfd\"/\"openshift-service-ca.crt\""
Jul 26 19:05:26.878294 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.878273    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fjlfd/must-gather-mmb5b"]
Jul 26 19:05:26.921097 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.921068    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/be9b56e7-efa0-4b80-b671-ec600bb61fa7-must-gather-output\") pod \"must-gather-mmb5b\" (UID: \"be9b56e7-efa0-4b80-b671-ec600bb61fa7\") " pod="openshift-must-gather-fjlfd/must-gather-mmb5b"
Jul 26 19:05:26.921223 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:26.921125    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w4bn\" (UniqueName: \"kubernetes.io/projected/be9b56e7-efa0-4b80-b671-ec600bb61fa7-kube-api-access-9w4bn\") pod \"must-gather-mmb5b\" (UID: \"be9b56e7-efa0-4b80-b671-ec600bb61fa7\") " pod="openshift-must-gather-fjlfd/must-gather-mmb5b"
Jul 26 19:05:27.022501 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:27.022463    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/be9b56e7-efa0-4b80-b671-ec600bb61fa7-must-gather-output\") pod \"must-gather-mmb5b\" (UID: \"be9b56e7-efa0-4b80-b671-ec600bb61fa7\") " pod="openshift-must-gather-fjlfd/must-gather-mmb5b"
Jul 26 19:05:27.022676 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:27.022518    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4bn\" (UniqueName: \"kubernetes.io/projected/be9b56e7-efa0-4b80-b671-ec600bb61fa7-kube-api-access-9w4bn\") pod \"must-gather-mmb5b\" (UID: \"be9b56e7-efa0-4b80-b671-ec600bb61fa7\") " pod="openshift-must-gather-fjlfd/must-gather-mmb5b"
Jul 26 19:05:27.022845 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:27.022825    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/be9b56e7-efa0-4b80-b671-ec600bb61fa7-must-gather-output\") pod \"must-gather-mmb5b\" (UID: \"be9b56e7-efa0-4b80-b671-ec600bb61fa7\") " pod="openshift-must-gather-fjlfd/must-gather-mmb5b"
Jul 26 19:05:27.041887 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:27.041861    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w4bn\" (UniqueName: \"kubernetes.io/projected/be9b56e7-efa0-4b80-b671-ec600bb61fa7-kube-api-access-9w4bn\") pod \"must-gather-mmb5b\" (UID: \"be9b56e7-efa0-4b80-b671-ec600bb61fa7\") " pod="openshift-must-gather-fjlfd/must-gather-mmb5b"
Jul 26 19:05:27.175374 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:27.175342    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fjlfd/must-gather-mmb5b"
Jul 26 19:05:27.308010 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:27.307904    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fjlfd/must-gather-mmb5b"]
Jul 26 19:05:27.310800 ip-10-0-138-70 kubenswrapper[2559]: W0726 19:05:27.310771    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe9b56e7_efa0_4b80_b671_ec600bb61fa7.slice/crio-aac7ea94261f03fa608640a59582671be45f087fc0854014baf6bdaadc2ab171 WatchSource:0}: Error finding container aac7ea94261f03fa608640a59582671be45f087fc0854014baf6bdaadc2ab171: Status 404 returned error can't find the container with id aac7ea94261f03fa608640a59582671be45f087fc0854014baf6bdaadc2ab171
Jul 26 19:05:27.312689 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:27.312670    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 26 19:05:27.995369 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:27.995326    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fjlfd/must-gather-mmb5b" event={"ID":"be9b56e7-efa0-4b80-b671-ec600bb61fa7","Type":"ContainerStarted","Data":"aac7ea94261f03fa608640a59582671be45f087fc0854014baf6bdaadc2ab171"}
Jul 26 19:05:29.004424 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:29.004381    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fjlfd/must-gather-mmb5b" event={"ID":"be9b56e7-efa0-4b80-b671-ec600bb61fa7","Type":"ContainerStarted","Data":"1c126db0f9bcc08be4f2be644a036eb2201c81382b05d9b138ed66d2a895282f"}
Jul 26 19:05:29.004936 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:29.004446    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fjlfd/must-gather-mmb5b" event={"ID":"be9b56e7-efa0-4b80-b671-ec600bb61fa7","Type":"ContainerStarted","Data":"11bf2d4f69177a7a26622211c49356532c1081da71cc69cf100b4709c7ba57b7"}
Jul 26 19:05:29.022773 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:29.022527    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fjlfd/must-gather-mmb5b" podStartSLOduration=2.271250944 podStartE2EDuration="3.022506408s" totalImagesPullingTime="751.255464ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-26 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:05:29.022444307 +0000 UTC m=+3200.346966405" watchObservedRunningTime="2026-07-26 19:05:29.022506408 +0000 UTC m=+3200.347028508"
Jul 26 19:05:29.656489 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:29.656458    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n4ng2_b28a6ca5-8e3c-4d29-8ed5-a684e40f6f0b/global-pull-secret-syncer/0.log"
Jul 26 19:05:29.801228 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:29.801191    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hm6m9_e7807e6d-3df2-4084-a6ea-a05859dd6eda/konnectivity-agent/0.log"
Jul 26 19:05:29.920816 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:29.920782    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-70.ec2.internal_a2a497afdef47058f3109e35eaa2548a/haproxy/0.log"
Jul 26 19:05:35.356660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:35.356570    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-spb9v_504040e5-5cb4-4fc9-a037-ed70147096b7/kuadrant-console-plugin/0.log"
Jul 26 19:05:35.642688 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:35.642439    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-snpqf_a3606262-f9fa-4397-9d5c-dd4521f2c146/manager/0.log"
Jul 26 19:05:37.437041 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.436995    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dd18ce3-429f-49d0-8397-2a369659fe31/alertmanager/0.log"
Jul 26 19:05:37.462489 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.462397    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dd18ce3-429f-49d0-8397-2a369659fe31/config-reloader/0.log"
Jul 26 19:05:37.506402 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.506370    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dd18ce3-429f-49d0-8397-2a369659fe31/kube-rbac-proxy-web/0.log"
Jul 26 19:05:37.529353 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.529328    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dd18ce3-429f-49d0-8397-2a369659fe31/kube-rbac-proxy/0.log"
Jul 26 19:05:37.556941 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.556908    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dd18ce3-429f-49d0-8397-2a369659fe31/kube-rbac-proxy-metric/0.log"
Jul 26 19:05:37.584208 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.584174    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dd18ce3-429f-49d0-8397-2a369659fe31/prom-label-proxy/0.log"
Jul 26 19:05:37.635541 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.635468    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3dd18ce3-429f-49d0-8397-2a369659fe31/init-config-reloader/0.log"
Jul 26 19:05:37.703997 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.703964    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-8rg26_26ef4de6-17cb-4fc3-93cf-2c3e724b3f49/kube-state-metrics/0.log"
Jul 26 19:05:37.728365 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.728284    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-8rg26_26ef4de6-17cb-4fc3-93cf-2c3e724b3f49/kube-rbac-proxy-main/0.log"
Jul 26 19:05:37.755361 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.755335    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-8rg26_26ef4de6-17cb-4fc3-93cf-2c3e724b3f49/kube-rbac-proxy-self/0.log"
Jul 26 19:05:37.787679 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.787648    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c9bd8f97f-6d9n2_16d4c49e-1ab3-4773-9345-937697b92a9b/metrics-server/0.log"
Jul 26 19:05:37.818105 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:37.818060    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-7zz4x_c45df73a-c600-4d13-9694-a5414a3e543e/monitoring-plugin/0.log"
Jul 26 19:05:38.031315 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.031242    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sp82r_7759bf11-07c1-4314-8133-74c3e58595fe/node-exporter/0.log"
Jul 26 19:05:38.054986 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.054953    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sp82r_7759bf11-07c1-4314-8133-74c3e58595fe/kube-rbac-proxy/0.log"
Jul 26 19:05:38.079020 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.078989    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sp82r_7759bf11-07c1-4314-8133-74c3e58595fe/init-textfile/0.log"
Jul 26 19:05:38.108109 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.108082    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-lcnwr_63bf1d7f-f67b-4a56-a194-1cc4e0b891ee/kube-rbac-proxy-main/0.log"
Jul 26 19:05:38.134158 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.134128    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-lcnwr_63bf1d7f-f67b-4a56-a194-1cc4e0b891ee/kube-rbac-proxy-self/0.log"
Jul 26 19:05:38.159533 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.159498    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-lcnwr_63bf1d7f-f67b-4a56-a194-1cc4e0b891ee/openshift-state-metrics/0.log"
Jul 26 19:05:38.207104 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.207067    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b/prometheus/0.log"
Jul 26 19:05:38.231217 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.231185    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b/config-reloader/0.log"
Jul 26 19:05:38.258436 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.258406    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b/thanos-sidecar/0.log"
Jul 26 19:05:38.283594 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.283526    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b/kube-rbac-proxy-web/0.log"
Jul 26 19:05:38.316297 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.316272    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b/kube-rbac-proxy/0.log"
Jul 26 19:05:38.342060 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.342031    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b/kube-rbac-proxy-thanos/0.log"
Jul 26 19:05:38.365702 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.365676    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_280e3f0a-585c-4a7f-9ce1-0e1c4f0bd78b/init-config-reloader/0.log"
Jul 26 19:05:38.396194 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.396166    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-44p5m_7748711a-cb96-4c24-b83c-06b923d0a868/prometheus-operator/0.log"
Jul 26 19:05:38.418060 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.418029    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-44p5m_7748711a-cb96-4c24-b83c-06b923d0a868/kube-rbac-proxy/0.log"
Jul 26 19:05:38.574300 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.574207    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8496b7b7c8-9dxhx_4089ec57-7c50-4c25-9921-d575c935d3e2/thanos-query/0.log"
Jul 26 19:05:38.607795 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.607765    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8496b7b7c8-9dxhx_4089ec57-7c50-4c25-9921-d575c935d3e2/kube-rbac-proxy-web/0.log"
Jul 26 19:05:38.640175 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.640133    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8496b7b7c8-9dxhx_4089ec57-7c50-4c25-9921-d575c935d3e2/kube-rbac-proxy/0.log"
Jul 26 19:05:38.674784 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.674755    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8496b7b7c8-9dxhx_4089ec57-7c50-4c25-9921-d575c935d3e2/prom-label-proxy/0.log"
Jul 26 19:05:38.711870 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.711836    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8496b7b7c8-9dxhx_4089ec57-7c50-4c25-9921-d575c935d3e2/kube-rbac-proxy-rules/0.log"
Jul 26 19:05:38.746798 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:38.746759    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8496b7b7c8-9dxhx_4089ec57-7c50-4c25-9921-d575c935d3e2/kube-rbac-proxy-metrics/0.log"
Jul 26 19:05:39.486664 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.486628    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"]
Jul 26 19:05:39.491007 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.490981    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.493855 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.493827    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fjlfd\"/\"default-dockercfg-shpr9\""
Jul 26 19:05:39.502913 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.502658    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"]
Jul 26 19:05:39.564720 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.564341    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-podres\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.564720 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.564457    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-proc\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.564720 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.564490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-lib-modules\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.564720 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.564540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcvfj\" (UniqueName: \"kubernetes.io/projected/3869b64e-e72e-4f6a-ae31-846c167aaf04-kube-api-access-wcvfj\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.564720 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.564599    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-sys\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.665187 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.665149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcvfj\" (UniqueName: \"kubernetes.io/projected/3869b64e-e72e-4f6a-ae31-846c167aaf04-kube-api-access-wcvfj\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.665660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.665237    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-sys\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.665660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.665304    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-podres\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.665660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.665336    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-sys\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.665660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.665423    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-proc\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.665660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.665440    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-podres\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.665660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.665460    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-lib-modules\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.665660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.665478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-proc\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.665660 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.665632    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3869b64e-e72e-4f6a-ae31-846c167aaf04-lib-modules\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.674833 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.674808    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcvfj\" (UniqueName: \"kubernetes.io/projected/3869b64e-e72e-4f6a-ae31-846c167aaf04-kube-api-access-wcvfj\") pod \"perf-node-gather-daemonset-8jzmw\" (UID: \"3869b64e-e72e-4f6a-ae31-846c167aaf04\") " pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.805231 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.805147    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:39.987257 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:39.987227    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"]
Jul 26 19:05:40.088636 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:40.087729    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw" event={"ID":"3869b64e-e72e-4f6a-ae31-846c167aaf04","Type":"ContainerStarted","Data":"23e1e47aeabb225cc2f001076393dd577dbb7b14b396ff4e58601efdc6741991"}
Jul 26 19:05:41.096669 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:41.096611    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw" event={"ID":"3869b64e-e72e-4f6a-ae31-846c167aaf04","Type":"ContainerStarted","Data":"414104d3248a505156810fecd204bbc677d30d8e0f998da377dc294e1476cd7a"}
Jul 26 19:05:41.097079 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:41.096742    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:41.114958 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:41.114909    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw" podStartSLOduration=2.114892812 podStartE2EDuration="2.114892812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-26 19:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-26 19:05:41.113739304 +0000 UTC m=+3212.438261424" watchObservedRunningTime="2026-07-26 19:05:41.114892812 +0000 UTC m=+3212.439414908"
Jul 26 19:05:42.727842 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:42.727808    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ndlbj_525f9463-314f-402a-8f92-ea019bd5d1da/dns/0.log"
Jul 26 19:05:42.750538 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:42.750508    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ndlbj_525f9463-314f-402a-8f92-ea019bd5d1da/kube-rbac-proxy/0.log"
Jul 26 19:05:42.831691 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:42.831661    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-54nt6_943739a7-45a0-444f-86dd-f0f5722ee83d/dns-node-resolver/0.log"
Jul 26 19:05:43.480392 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:43.480340    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5bt9b_35a8bada-5aff-496e-ad10-c359a3e6e416/node-ca/0.log"
Jul 26 19:05:44.715279 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:44.715246    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c7c67d4f6-b8krv_89307c3c-e972-476b-963c-4df2e54d2f92/kube-auth-proxy/0.log"
Jul 26 19:05:45.807092 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:45.807056    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lvz4w_08f03152-02ec-4d03-9cda-5fd6fb479ade/serve-healthcheck-canary/0.log"
Jul 26 19:05:46.485345 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:46.485310    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktklq_f93b966c-2271-4877-abdd-5af88a37caea/kube-rbac-proxy/0.log"
Jul 26 19:05:46.510304 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:46.510275    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktklq_f93b966c-2271-4877-abdd-5af88a37caea/exporter/0.log"
Jul 26 19:05:46.534263 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:46.534233    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ktklq_f93b966c-2271-4877-abdd-5af88a37caea/extractor/0.log"
Jul 26 19:05:47.114151 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:47.114117    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fjlfd/perf-node-gather-daemonset-8jzmw"
Jul 26 19:05:49.404105 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:49.404073    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8494c84d99-k6zhp_402e9e41-e678-4722-b6a2-13a0af69febe/manager/0.log"
Jul 26 19:05:49.707545 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:49.707511    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-mqvph_05604bb7-666e-4786-a350-bea356d5e5a5/manager/0.log"
Jul 26 19:05:49.734350 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:49.734327    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-758747dbfc-mqvph_05604bb7-666e-4786-a350-bea356d5e5a5/copy-manifests/0.log"
Jul 26 19:05:50.059975 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:50.059882    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c7c67d4f6-b8krv_89307c3c-e972-476b-963c-4df2e54d2f92/kube-auth-proxy/0.log"
Jul 26 19:05:51.018013 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:51.017975    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5b76bfd8c9-nhhr6_0e3b51c7-ff6b-47d5-8c03-158ed9c0bcd5/manager/0.log"
Jul 26 19:05:51.047921 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:51.047883    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-pdkf5_38d54870-7857-45b9-879a-7e9c81a82a56/openshift-lws-operator/0.log"
Jul 26 19:05:51.782347 ip-10-0-138-70 kubenswrapper[2559]: I0726 19:05:51.782310    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c7c67d4f6-b8krv_89307c3c-e972-476b-963c-4df2e54d2f92/kube-auth-proxy/0.log"