Sep 11 13:55:40.486590 ip-10-0-129-195 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 13:55:40.486756 ip-10-0-129-195 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 13:55:40.486855 ip-10-0-129-195 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 13:55:40.487121 ip-10-0-129-195 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 13:55:49.893870 ip-10-0-129-195 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 11 13:55:49.893882 ip-10-0-129-195 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 96ea502866a4440388cb102410f5eec1 --
Sep 11 13:57:29.536918 ip-10-0-129-195 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 13:57:29.983519 ip-10-0-129-195 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:57:29.983519 ip-10-0-129-195 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 13:57:29.983519 ip-10-0-129-195 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:57:29.983519 ip-10-0-129-195 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 13:57:29.983519 ip-10-0-129-195 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:57:29.986361 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.986276    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 13:57:29.991336 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991321    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:57:29.991336 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991335    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991339    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991343    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991346    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991349    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991352    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991355    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991359    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991362    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991364    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991368    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991370    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991373    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991376    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991382    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991385    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991387    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991391    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991394    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:57:29.991398 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991397    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991399    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991402    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991405    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991409    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991412    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991415    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991419    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991423    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991426    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991429    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991432    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991435    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991437    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991440    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991443    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991446    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991448    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991451    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:57:29.991862 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991453    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991456    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991458    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991461    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991464    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991466    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991468    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991471    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991473    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991476    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991478    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991481    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991483    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991487    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991489    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991493    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991496    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991498    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991501    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:57:29.992326 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991504    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991506    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991511    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991515    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991518    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991520    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991523    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991526    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991528    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991531    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991534    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991537    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991539    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991543    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991546    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991548    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991551    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991556    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991559    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991562    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:57:29.992801 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991565    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991568    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991571    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991573    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991576    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991578    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991582    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991585    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991588    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991591    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991593    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991596    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991598    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991601    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991604    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991606    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991609    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991612    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991614    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991617    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:57:29.993279 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.991619    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:57:29.994107 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994094    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:57:29.994107 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994106    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994110    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994113    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994116    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994120    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994123    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994125    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994128    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994133    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994136    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994140    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994143    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994146    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994148    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994151    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994154    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994156    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994159    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994162    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:57:29.994162 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994164    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994168    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994171    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994174    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994177    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994179    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994182    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994185    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994187    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994190    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994192    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994195    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994198    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994200    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994203    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994205    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994208    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994210    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994215    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994217    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:57:29.994642 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994220    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994222    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994225    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994227    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994230    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994232    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994234    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994237    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994240    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994242    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994245    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994247    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994250    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994252    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994255    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994258    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994263    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994267    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994270    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:57:29.995150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994273    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994276    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994278    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994281    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994284    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994286    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994289    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994292    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994296    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994299    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994302    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994305    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994314    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994317    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994319    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994322    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994325    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994327    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994329    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994332    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:57:29.995620 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994334    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994337    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994339    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994342    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994344    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994347    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994349    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994352    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994355    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994357    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994360    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994362    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994365    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994367    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994370    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994372    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994375    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994377    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994380    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994382    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:57:29.996123 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994799    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994804    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994806    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994809    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994818    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994821    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994823    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994826    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994829    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994832    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994834    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994837    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994840    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994843    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994845    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994848    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994850    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994853    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994855    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994860    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:57:29.996639 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994863    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994866    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994868    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994871    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994873    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994876    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994878    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994881    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994883    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994886    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994889    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994891    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994893    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994896    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994898    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994901    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994903    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994911    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994914    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994917    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:57:29.997129 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994926    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994929    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994932    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994934    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994937    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994940    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994943    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994946    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994949    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994951    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994954    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994959    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994962    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994964    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994967    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994969    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994972    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994974    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994977    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:57:29.997660 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994981    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994984    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994987    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994990    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994992    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994996    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.994999    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995002    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995004    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995007    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995009    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995020    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995023    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995025    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995028    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995030    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995033    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995035    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995037    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995040    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:57:29.998112 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995042    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995045    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995048    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995051    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995055    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995057    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995060    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995062    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995065    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995067    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995070    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995073    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995075    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995077    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995080    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995084    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995087    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995091    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995093    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995096    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:57:29.998587 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995170    2564 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995207    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995219    2564 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995225    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995230    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995233    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995247    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995252    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995255    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995258    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995261    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995265    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995267    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995270    2564 flags.go:64] FLAG: --cgroup-root=""
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995273    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995276    2564 flags.go:64] FLAG: --client-ca-file=""
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995280    2564 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995283    2564 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995288    2564 flags.go:64] FLAG: --cluster-domain=""
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995291    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995294    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995297    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995300    2564 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995304    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 13:57:29.999151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995307    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995310    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995313    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995316    2564 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995320    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995322    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995325    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995328    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995332    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995335    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995338    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995341    2564 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995344    2564 flags.go:64] FLAG: --enable-server="true"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995347    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995351    2564 flags.go:64] FLAG: --event-burst="100"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995354    2564 flags.go:64] FLAG: --event-qps="50"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995357    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995360    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995362    2564 flags.go:64] FLAG: --eviction-hard=""
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995366    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995369    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995372    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995375    2564 flags.go:64] FLAG: --eviction-soft=""
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995377    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995381    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 13:57:29.999734 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995384    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995387    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995390    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995392    2564 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995395    2564 flags.go:64] FLAG: --feature-gates=""
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995399    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995402    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995405    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995408    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995411    2564 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995414    2564 flags.go:64] FLAG: --help="false"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995417    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995420    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995423    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995425    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995429    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995432    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995435    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995438    2564 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995440    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995443    2564 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995447    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995450    2564 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995453    2564 flags.go:64] FLAG: --kube-reserved=""
Sep 11 13:57:30.000428 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995456    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995458    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995461    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995464    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995467    2564 flags.go:64] FLAG: --lock-file=""
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995471    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995474    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995478    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995483    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995486    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995489    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995492    2564 flags.go:64] FLAG: --logging-format="text"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995495    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995498    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995501    2564 flags.go:64] FLAG: --manifest-url=""
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995504    2564 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995508    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995511    2564 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995515    2564 flags.go:64] FLAG: --max-pods="110"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995519    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995522    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995524    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995527    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995531    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995534    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 13:57:30.001026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995537    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995543    2564 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995546    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995549    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995552    2564 flags.go:64] FLAG: --pod-cidr=""
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995556    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995561    2564 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995564    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995567    2564 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995570    2564 flags.go:64] FLAG: --port="10250"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995573    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995576    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-030162b022b9399e0"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995579    2564 flags.go:64] FLAG: --qos-reserved=""
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995585    2564 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995588    2564 flags.go:64] FLAG: --register-node="true"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995591    2564 flags.go:64] FLAG: --register-with-taints=""
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995595    2564 flags.go:64] FLAG: --registry-burst="10"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995598    2564 flags.go:64] FLAG: --registry-qps="5"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995601    2564 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995604    2564 flags.go:64] FLAG: --reserved-memory=""
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995607    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995611    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995637    2564 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995642    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995646    2564 flags.go:64] FLAG: --runonce="false"
Sep 11 13:57:30.001620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995649    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995652    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995656    2564 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995659    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995662    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995665    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995668    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995671    2564 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995674    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995677    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995680    2564 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995683    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995686    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995689    2564 flags.go:64] FLAG: --system-cgroups=""
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995692    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995698    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995700    2564 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995703    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995708    2564 flags.go:64] FLAG: --tls-min-version=""
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995710    2564 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995714    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995719    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995722    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995725    2564 flags.go:64] FLAG: --v="2"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995729    2564 flags.go:64] FLAG: --version="false"
Sep 11 13:57:30.002236 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995733    2564 flags.go:64] FLAG: --vmodule=""
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995736    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.995740    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995900    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995904    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995907    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995910    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995914    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995917    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995919    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995922    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995925    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995927    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995930    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995932    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995935    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995937    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995940    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995942    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995945    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:57:30.002903 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995947    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995955    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995958    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995961    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995963    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995966    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995969    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995972    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995976    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995979    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995987    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995990    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995993    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995996    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.995998    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996001    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996003    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996006    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996008    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:57:30.003424 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996011    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996015    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996018    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996021    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996024    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996027    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996030    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996032    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996035    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996038    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996040    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996043    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996046    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996048    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996051    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996060    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996063    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996065    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996068    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996070    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:57:30.003919 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996074    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996079    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996081    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996084    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996087    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996097    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996100    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996102    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996105    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996108    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996112    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996115    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996119    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996122    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996125    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996128    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996131    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996134    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996137    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:57:30.004417 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996141    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996144    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996147    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996149    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996152    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996155    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996157    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996160    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996163    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996171    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996174    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996176    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996179    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996183    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996187    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996190    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996192    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996195    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996198    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996200    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:57:30.004898 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996203    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:57:30.005383 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996206    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:57:30.005383 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996208    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:57:30.005383 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:29.996211    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:57:30.005383 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:29.996980    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:57:30.005383 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.004497    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 13:57:30.005383 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.004510    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.005400    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.005407    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005461    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005466    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005469    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005472    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005477    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005481    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005485    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005488    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005491    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005493    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005496    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005498    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005501    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005504    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005507    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005509    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005512    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:57:30.005537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005515    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005517    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005520    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005522    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005525    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005528    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005530    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005533    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005535    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005538    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005540    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005543    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005547    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005550    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005553    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005556    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005558    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005561    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005563    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005566    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:57:30.006150 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005569    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005571    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005574    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005576    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005579    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005581    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005584    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005586    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005589    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005591    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005594    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005596    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005599    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005602    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005605    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005608    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005610    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005613    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005615    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005618    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:57:30.006837 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005620    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005623    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005639    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005642    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005644    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005647    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005650    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005652    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005655    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005658    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005660    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005663    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005665    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005668    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005671    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005673    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005677    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005681    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005684    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:57:30.007591 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005687    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005689    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005692    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005695    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005698    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005701    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005704    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005706    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005709    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005711    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005714    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005716    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005719    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005722    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005726    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005729    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005737    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005740    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005744    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:57:30.008077 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005747    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005750    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005753    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005755    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.005760    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005865    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005869    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005872    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005875    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005878    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005882    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005884    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005887    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005889    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005892    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:57:30.008537 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005894    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005897    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005899    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005902    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005904    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005907    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005910    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005912    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005915    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005917    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005920    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005922    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005925    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005927    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005929    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005932    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005934    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005937    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005940    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005942    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:57:30.008935 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005944    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005947    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005950    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005953    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005957    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005959    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005962    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005964    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005967    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005969    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005972    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005974    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005977    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005979    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005982    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005984    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005988    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005991    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005994    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:57:30.009472 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.005997    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006000    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006002    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006005    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006007    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006010    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006012    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006015    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006017    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006020    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006022    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006025    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006028    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006030    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006033    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006035    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006038    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006040    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006043    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006045    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:57:30.010012 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006047    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006050    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006052    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006055    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006057    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006061    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006064    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006067    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006070    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006072    2564 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006075    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006078    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006081    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006083    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006086    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006088    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006091    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006093    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006095    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006098    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:57:30.010666 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006100    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006103    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006105    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006108    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006110    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006113    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006116    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006118    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006121    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:30.006123    2564 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.006128    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.006219    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.009029    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.010197    2564 server.go:1019] "Starting client certificate rotation"
Sep 11 13:57:30.011247 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.010305    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:57:30.011590 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.010344    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:57:30.013070 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.013057    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:57:30.014259 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.014244    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:57:30.022464 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.022447    2564 log.go:25] "Validated CRI v1 runtime API"
Sep 11 13:57:30.025698 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.025684    2564 log.go:25] "Validated CRI v1 image API"
Sep 11 13:57:30.026497 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.026483    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 13:57:30.032738 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.032721    2564 fs.go:135] Filesystem UUIDs: map[0d526b31-9fec-4db6-8f62-21b2a60022da:/dev/nvme0n1p3 79713eb1-2729-49e4-b6d7-0f22030d5c25:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 11 13:57:30.032790 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.032737    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 13:57:30.038745 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.038634    2564 manager.go:216] Machine: {Timestamp:2026-09-11 13:57:30.037238199 +0000 UTC m=+0.374813754 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101069 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f078416795f4af6887f47957ca299 SystemUUID:ec2f0784-1679-5f4a-f688-7f47957ca299 BootID:96ea5028-66a4-4403-88cb-102410f5eec1 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a0:7b:99:53:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a0:7b:99:53:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:b3:9d:34:82:cf Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 13:57:30.038745 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.038739    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 13:57:30.038858 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.038808    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 13:57:30.042395 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.042372    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 13:57:30.042522 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.042398    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-195.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 13:57:30.043206 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.043196    2564 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 13:57:30.043237 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.043208    2564 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 13:57:30.043237 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.043216    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:57:30.043524 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.043508    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:57:30.043949 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.043935    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:57:30.045201 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.045192    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 13:57:30.049862 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.049850    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:57:30.049975 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.049966    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 13:57:30.050008 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.049998    2564 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 13:57:30.050035 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.050011    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 13:57:30.050035 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.050022    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 13:57:30.050035 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.050031    2564 kubelet.go:398] "Adding apiserver pod source"
Sep 11 13:57:30.050150 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.050038    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 13:57:30.050431 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.050417    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:57:30.050493 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.050437    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:57:30.054617 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.054592    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 13:57:30.057089 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.057073    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 13:57:30.057163 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.057107    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 13:57:30.059794 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059780    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059801    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059807    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059813    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059822    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059827    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059841    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059846    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059853    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059859    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059867    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 13:57:30.059875 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.059872    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 13:57:30.060791 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.060781    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 13:57:30.060791 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.060790    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 13:57:30.062163 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.062138    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-195.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 13:57:30.065157 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.065136    2564 server.go:1268] "Started kubelet"
Sep 11 13:57:30.065238 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.065217    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 13:57:30.065295 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.065264    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 13:57:30.065911 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.065871    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 13:57:30.065911 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.065904    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-195.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 13:57:30.066040 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.065971    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 13:57:30.065922 ip-10-0-129-195 systemd[1]: Started Kubernetes Kubelet.
Sep 11 13:57:30.068451 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.067398    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 13:57:30.070245 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.070227    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 13:57:30.070329 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.070272    2564 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 13:57:30.074529 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.074510    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 13:57:30.075909 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.075200    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 13:57:30.076051 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.076033    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.076118 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.076061    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 13:57:30.076118 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.076078    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 13:57:30.076118 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.074938    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-195.ec2.internal.18d448d94a54ec84  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-195.ec2.internal,UID:ip-10-0-129-195.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-195.ec2.internal,},FirstTimestamp:2026-09-11 13:57:30.065112196 +0000 UTC m=+0.402687746,LastTimestamp:2026-09-11 13:57:30.065112196 +0000 UTC m=+0.402687746,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-195.ec2.internal,}"
Sep 11 13:57:30.076265 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.076189    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 13:57:30.076399 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.076383    2564 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 13:57:30.076460 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.076396    2564 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 13:57:30.077979 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.077961    2564 factory.go:55] Registering systemd factory
Sep 11 13:57:30.077979 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.077977    2564 factory.go:223] Registration of the systemd container factory successfully
Sep 11 13:57:30.078212 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.078198    2564 factory.go:153] Registering CRI-O factory
Sep 11 13:57:30.078268 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.078231    2564 factory.go:223] Registration of the crio container factory successfully
Sep 11 13:57:30.078325 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.078282    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 13:57:30.078325 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.078300    2564 factory.go:103] Registering Raw factory
Sep 11 13:57:30.078325 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.078314    2564 manager.go:1196] Started watching for new ooms in manager
Sep 11 13:57:30.078785 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.078773    2564 manager.go:349] Starting recovery of all containers
Sep 11 13:57:30.079448 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.079421    2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-195.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 13:57:30.079530 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.079512    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 13:57:30.084529 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.084408    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 13:57:30.091537 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.091516    2564 manager.go:354] Recovery completed
Sep 11 13:57:30.098155 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.098140    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:57:30.100360 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.100325    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:57:30.100423 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.100369    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:57:30.100423 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.100386    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:57:30.101002 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.100990    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 13:57:30.101002 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.101003    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 13:57:30.101134 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.101020    2564 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:57:30.101134 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.100993    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:57:30.101814 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.101799    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:57:30.101915 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.101840    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:57:30.101915 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.101852    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:57:30.102338 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.102261    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-195.ec2.internal.18d448d94c6ea567  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-195.ec2.internal,UID:ip-10-0-129-195.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-195.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-195.ec2.internal,},FirstTimestamp:2026-09-11 13:57:30.100352359 +0000 UTC m=+0.437927913,LastTimestamp:2026-09-11 13:57:30.100352359 +0000 UTC m=+0.437927913,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-195.ec2.internal,}"
Sep 11 13:57:30.103229 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.103214    2564 policy_none.go:49] "None policy: Start"
Sep 11 13:57:30.103283 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.103233    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 13:57:30.103283 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.103246    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 13:57:30.105310 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.105295    2564 policy_none.go:47] "Start"
Sep 11 13:57:30.113024 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.113002    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m677n"
Sep 11 13:57:30.113776 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.112566    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-195.ec2.internal.18d448d94c6f069e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-195.ec2.internal,UID:ip-10-0-129-195.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-195.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-195.ec2.internal,},FirstTimestamp:2026-09-11 13:57:30.100377246 +0000 UTC m=+0.437952798,LastTimestamp:2026-09-11 13:57:30.100377246 +0000 UTC m=+0.437952798,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-195.ec2.internal,}"
Sep 11 13:57:30.119230 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.119215    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m677n"
Sep 11 13:57:30.124681 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.124597    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-195.ec2.internal.18d448d94c6f3db4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-195.ec2.internal,UID:ip-10-0-129-195.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-195.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-195.ec2.internal,},FirstTimestamp:2026-09-11 13:57:30.100391348 +0000 UTC m=+0.437966899,LastTimestamp:2026-09-11 13:57:30.100391348 +0000 UTC m=+0.437966899,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-195.ec2.internal,}"
Sep 11 13:57:30.139224 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.139209    2564 manager.go:337] "Starting Device Plugin manager"
Sep 11 13:57:30.139313 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.139246    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 13:57:30.139313 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.139260    2564 server.go:87] "Starting device plugin registration server"
Sep 11 13:57:30.139560 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.139547    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 13:57:30.139620 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.139561    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 13:57:30.139696 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.139653    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 13:57:30.139742 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.139730    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 13:57:30.139742 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.139739    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 13:57:30.140245 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.140229    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 13:57:30.140318 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.140267    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.167234 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.167212    2564 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2598 into the system container \"/system.slice\": <nil>"
Sep 11 13:57:30.239751 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.239670    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:57:30.240698 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.240680    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:57:30.240815 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.240709    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:57:30.240815 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.240721    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:57:30.240815 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.240751    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.250793 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.250774    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.250882 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.250796    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-195.ec2.internal\": node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.268663 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.268644    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 13:57:30.272787 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.268689    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 13:57:30.272787 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.268716    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 13:57:30.272787 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.268750    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 13:57:30.272787 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.271037    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:57:30.284804 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.284789    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.369539 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.369509    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal"]
Sep 11 13:57:30.369642 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.369578    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:57:30.371398 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.371383    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:57:30.371488 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.371415    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:57:30.371488 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.371430    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:57:30.373894 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.373879    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:57:30.374010 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.373994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.374056 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.374021    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:57:30.374599 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.374587    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:57:30.374599 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.374592    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:57:30.374750 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.374611    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:57:30.374750 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.374616    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:57:30.374750 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.374621    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:57:30.374750 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.374647    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:57:30.376849 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.376836    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.376891 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.376859    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:57:30.377217 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.377187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb4517e21fa1b4a954b46d170bcf0a3a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal\" (UID: \"eb4517e21fa1b4a954b46d170bcf0a3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.377319 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.377228    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb4517e21fa1b4a954b46d170bcf0a3a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal\" (UID: \"eb4517e21fa1b4a954b46d170bcf0a3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.377319 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.377254    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a193ec36d49eb843a04da4a94012319-config\") pod \"kube-apiserver-proxy-ip-10-0-129-195.ec2.internal\" (UID: \"1a193ec36d49eb843a04da4a94012319\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.377513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.377491    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:57:30.377600 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.377524    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:57:30.377600 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.377547    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:57:30.385682 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.385664    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.390128 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.390113    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-195.ec2.internal\" not found" node="ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.393372 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.393357    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-195.ec2.internal\" not found" node="ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.477382 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.477354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb4517e21fa1b4a954b46d170bcf0a3a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal\" (UID: \"eb4517e21fa1b4a954b46d170bcf0a3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.477382 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.477384    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb4517e21fa1b4a954b46d170bcf0a3a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal\" (UID: \"eb4517e21fa1b4a954b46d170bcf0a3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.477555 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.477407    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a193ec36d49eb843a04da4a94012319-config\") pod \"kube-apiserver-proxy-ip-10-0-129-195.ec2.internal\" (UID: \"1a193ec36d49eb843a04da4a94012319\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.477555 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.477449    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1a193ec36d49eb843a04da4a94012319-config\") pod \"kube-apiserver-proxy-ip-10-0-129-195.ec2.internal\" (UID: \"1a193ec36d49eb843a04da4a94012319\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.477555 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.477452    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb4517e21fa1b4a954b46d170bcf0a3a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal\" (UID: \"eb4517e21fa1b4a954b46d170bcf0a3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.477555 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.477478    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb4517e21fa1b4a954b46d170bcf0a3a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal\" (UID: \"eb4517e21fa1b4a954b46d170bcf0a3a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.486477 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.486461    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.587365 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.587292    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.687811 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.687777    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.692948 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.692931    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.697007 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.696990    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.788836 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.788802    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.889316 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:30.889248    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-195.ec2.internal\" not found"
Sep 11 13:57:30.939260 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.939236    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:57:30.976884 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.976861    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:30.993330 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.993313    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:57:30.995418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:30.995404    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal"
Sep 11 13:57:31.005202 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.005187    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:57:31.009605 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.009588    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 13:57:31.009724 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.009708    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 13:57:31.051689 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.051668    2564 apiserver.go:52] "Watching apiserver"
Sep 11 13:57:31.059901 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.059880    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 13:57:31.060250 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.060230    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-29mkt","openshift-ovn-kubernetes/ovnkube-node-zb9z7","kube-system/global-pull-secret-syncer-vz6bg","kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq","openshift-cluster-node-tuning-operator/tuned-6nc5n","openshift-dns/node-resolver-2zmjn","openshift-multus/multus-additional-cni-plugins-kf48t","openshift-multus/network-metrics-daemon-sjsjk","openshift-network-diagnostics/network-check-target-r48dd","kube-system/konnectivity-agent-ljzn7","openshift-image-registry/node-ca-hjsjr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal","openshift-multus/multus-fvqbb"]
Sep 11 13:57:31.063185 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.063169    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:31.063561 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.063530    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:31.066106 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.066084    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.068379 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.068359    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.068855 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.068837    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 13:57:31.068948 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.068876    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 13:57:31.068948 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.068885    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 13:57:31.069119 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.068985    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 13:57:31.069119 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.069013    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-d7p27\""
Sep 11 13:57:31.069119 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.069049    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 13:57:31.069290 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.069277    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 13:57:31.070422 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.070406    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 13:57:31.070657 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.070641    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 13:57:31.070796 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.070772    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 13:57:31.070796 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.070779    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.070796 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.070783    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 13:57:31.071072 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.070941    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 13:57:31.071164 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.071090    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qj52f\""
Sep 11 13:57:31.073136 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.073116    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 13:57:31.073209 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.073136    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 13:57:31.073300 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.073286    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.073370 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.073347    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qn4z6\""
Sep 11 13:57:31.073629 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.073615    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 13:57:31.074674 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.074659    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 13:57:31.075393 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.075380    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-s7ltf\""
Sep 11 13:57:31.075458 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.075442    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 13:57:31.075587 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.075574    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 13:57:31.075770 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.075755    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.075908 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.075893    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 13:57:31.077768 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.077751    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:57:31.078048 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.078033    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-skslg\""
Sep 11 13:57:31.078129 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.078036    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 13:57:31.078187 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.078161    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.080329 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-run-netns\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.080407 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080335    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-etc-openvswitch\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.080407 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/035c4d99-4e78-4b8b-a4ab-f0214f882c45-ovnkube-config\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.080407 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080388    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znqfk\" (UniqueName: \"kubernetes.io/projected/9c939c52-06f2-4030-846f-c187cbbdaec6-kube-api-access-znqfk\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.080551 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-sysctl-conf\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.080551 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-var-lib-kubelet\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.080551 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080485    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-etc-selinux\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.080551 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00dc4e9d-f4e7-41a6-9d88-3411a8df5400-host\") pod \"node-ca-hjsjr\" (UID: \"00dc4e9d-f4e7-41a6-9d88-3411a8df5400\") " pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.080779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e377a553-4efb-4b16-9679-b8c9ca1e3803-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.080779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080605    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-kubernetes\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.080779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080618    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.080779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080631    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-run\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.080779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080680    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-host\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.080779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080698    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cba141b-2eef-4463-89e5-87428f1036e1-host-slash\") pod \"iptables-alerter-29mkt\" (UID: \"7cba141b-2eef-4463-89e5-87428f1036e1\") " pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.080779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080715    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4sst\" (UniqueName: \"kubernetes.io/projected/035c4d99-4e78-4b8b-a4ab-f0214f882c45-kube-api-access-f4sst\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.080779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-tuned\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.080779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080760    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-cni-netd\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080798    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e377a553-4efb-4b16-9679-b8c9ca1e3803-cni-binary-copy\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080827    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-sys\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080852    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-systemd-units\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080865    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-log-socket\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080883    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-run-ovn-kubernetes\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080906    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-cni-bin\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdv7f\" (UniqueName: \"kubernetes.io/projected/01b43bc3-9185-4f02-8486-3668fac6aa3f-kube-api-access-mdv7f\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.080997    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e377a553-4efb-4b16-9679-b8c9ca1e3803-system-cni-dir\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081029    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7cba141b-2eef-4463-89e5-87428f1036e1-iptables-alerter-script\") pod \"iptables-alerter-29mkt\" (UID: \"7cba141b-2eef-4463-89e5-87428f1036e1\") " pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081046    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e377a553-4efb-4b16-9679-b8c9ca1e3803-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.081152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081075    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-sysctl-d\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01b43bc3-9185-4f02-8486-3668fac6aa3f-tmp\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/035c4d99-4e78-4b8b-a4ab-f0214f882c45-env-overrides\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-sys-fs\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081252    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-modprobe-d\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081283    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-systemd\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081306    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-device-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-sysconfig\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081349    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq9kd\" (UniqueName: \"kubernetes.io/projected/7cba141b-2eef-4463-89e5-87428f1036e1-kube-api-access-nq9kd\") pod \"iptables-alerter-29mkt\" (UID: \"7cba141b-2eef-4463-89e5-87428f1036e1\") " pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-kubelet\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081395    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-run-openvswitch\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081434    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081457    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-lib-modules\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081482    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/035c4d99-4e78-4b8b-a4ab-f0214f882c45-ovnkube-script-lib\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-socket-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081459    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kwkxt\""
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081514    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-node-log\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.081552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081530    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-registration-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081560    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081582    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/035c4d99-4e78-4b8b-a4ab-f0214f882c45-ovn-node-metrics-cert\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081615    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e377a553-4efb-4b16-9679-b8c9ca1e3803-cnibin\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081665    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-run-ovn\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081714    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e377a553-4efb-4b16-9679-b8c9ca1e3803-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081743    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081742    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhrrk\" (UniqueName: \"kubernetes.io/projected/e377a553-4efb-4b16-9679-b8c9ca1e3803-kube-api-access-jhrrk\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081787    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-run-systemd\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081853    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081876    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-var-lib-openvswitch\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081894    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/00dc4e9d-f4e7-41a6-9d88-3411a8df5400-serviceca\") pod \"node-ca-hjsjr\" (UID: \"00dc4e9d-f4e7-41a6-9d88-3411a8df5400\") " pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081951    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d424p\" (UniqueName: \"kubernetes.io/projected/00dc4e9d-f4e7-41a6-9d88-3411a8df5400-kube-api-access-d424p\") pod \"node-ca-hjsjr\" (UID: \"00dc4e9d-f4e7-41a6-9d88-3411a8df5400\") " pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081974    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 13:57:31.082228 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.081985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-slash\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.082818 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.082799    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 13:57:31.083127 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.083106    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:31.083217 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.083182    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:31.083217 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.083202    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lp7gp\""
Sep 11 13:57:31.083570 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.083552    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 13:57:31.085587 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.085572    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:31.087739 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.087725    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 13:57:31.088132 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.088116    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.088173 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.088155    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ltq2c\""
Sep 11 13:57:31.088209 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.088185    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 13:57:31.088209 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.088183    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:31.090371 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.090356    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.092935 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.092917    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:57:31.092935 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.092928    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 13:57:31.093142 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.093128    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mr5fm\""
Sep 11 13:57:31.124693 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.124630    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 13:52:30 +0000 UTC" deadline="2028-03-08 22:32:40.61178761 +0000 UTC"
Sep 11 13:57:31.124693 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.124691    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13064h35m9.487099588s"
Sep 11 13:57:31.124849 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.124745    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dw52c"
Sep 11 13:57:31.131388 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.131370    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dw52c"
Sep 11 13:57:31.176914 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.176895    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 13:57:31.183121 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183103    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-var-lib-kubelet\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.183272 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183127    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlzbm\" (UniqueName: \"kubernetes.io/projected/a8041769-d3ac-4348-9ce8-f77ed826aa8d-kube-api-access-qlzbm\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.183272 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183150    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00dc4e9d-f4e7-41a6-9d88-3411a8df5400-host\") pod \"node-ca-hjsjr\" (UID: \"00dc4e9d-f4e7-41a6-9d88-3411a8df5400\") " pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.183272 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e377a553-4efb-4b16-9679-b8c9ca1e3803-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.183272 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183196    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8041769-d3ac-4348-9ce8-f77ed826aa8d-cni-binary-copy\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.183272 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183219    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-hostroot\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.183272 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183246    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/00dc4e9d-f4e7-41a6-9d88-3411a8df5400-host\") pod \"node-ca-hjsjr\" (UID: \"00dc4e9d-f4e7-41a6-9d88-3411a8df5400\") " pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.183272 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183262    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-kubernetes\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183285    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-run\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183302    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-host\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cba141b-2eef-4463-89e5-87428f1036e1-host-slash\") pod \"iptables-alerter-29mkt\" (UID: \"7cba141b-2eef-4463-89e5-87428f1036e1\") " pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183335    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4sst\" (UniqueName: \"kubernetes.io/projected/035c4d99-4e78-4b8b-a4ab-f0214f882c45-kube-api-access-f4sst\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183386    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-host\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183399    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-kubernetes\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183415    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183422    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-run\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-tuned\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-cni-netd\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183397    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7cba141b-2eef-4463-89e5-87428f1036e1-host-slash\") pod \"iptables-alerter-29mkt\" (UID: \"7cba141b-2eef-4463-89e5-87428f1036e1\") " pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e377a553-4efb-4b16-9679-b8c9ca1e3803-cni-binary-copy\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183516    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-cni-netd\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.183536 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4ff248cb-b2cd-46e6-865f-c5d26df89969-konnectivity-ca\") pod \"konnectivity-agent-ljzn7\" (UID: \"4ff248cb-b2cd-46e6-865f-c5d26df89969\") " pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183549    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gq5g\" (UniqueName: \"kubernetes.io/projected/73e95838-8a95-4047-b739-0f740d140756-kube-api-access-6gq5g\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183577    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-sys\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183601    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-systemd-units\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183626    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-log-socket\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-sys\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183673    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-run-ovn-kubernetes\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183677    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-systemd-units\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-log-socket\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183692    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-cni-bin\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183725    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-run-ovn-kubernetes\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183727    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-cni-bin\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-etc-kubernetes\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183770    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdv7f\" (UniqueName: \"kubernetes.io/projected/01b43bc3-9185-4f02-8486-3668fac6aa3f-kube-api-access-mdv7f\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183788    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e377a553-4efb-4b16-9679-b8c9ca1e3803-system-cni-dir\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183804    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa-tmp-dir\") pod \"node-resolver-2zmjn\" (UID: \"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa\") " pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183826    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9953ef84-7f0d-4d91-8f62-735ee11380b3-kubelet-config\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.184263 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183837    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e377a553-4efb-4b16-9679-b8c9ca1e3803-system-cni-dir\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7cba141b-2eef-4463-89e5-87428f1036e1-iptables-alerter-script\") pod \"iptables-alerter-29mkt\" (UID: \"7cba141b-2eef-4463-89e5-87428f1036e1\") " pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e377a553-4efb-4b16-9679-b8c9ca1e3803-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4ff248cb-b2cd-46e6-865f-c5d26df89969-agent-certs\") pod \"konnectivity-agent-ljzn7\" (UID: \"4ff248cb-b2cd-46e6-865f-c5d26df89969\") " pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183933    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-system-cni-dir\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-sysctl-d\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.183981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01b43bc3-9185-4f02-8486-3668fac6aa3f-tmp\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184006    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/035c4d99-4e78-4b8b-a4ab-f0214f882c45-env-overrides\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-sys-fs\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184054    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-run-multus-certs\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184075    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-modprobe-d\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-systemd\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184129    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-sysctl-d\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184141    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-device-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184172    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e377a553-4efb-4b16-9679-b8c9ca1e3803-cni-binary-copy\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184206    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-sys-fs\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184227    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-device-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.184996 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-sysconfig\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq9kd\" (UniqueName: \"kubernetes.io/projected/7cba141b-2eef-4463-89e5-87428f1036e1-kube-api-access-nq9kd\") pod \"iptables-alerter-29mkt\" (UID: \"7cba141b-2eef-4463-89e5-87428f1036e1\") " pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-systemd\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-kubelet\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184321    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-modprobe-d\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184339    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-run-openvswitch\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184364    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9953ef84-7f0d-4d91-8f62-735ee11380b3-dbus\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e377a553-4efb-4b16-9679-b8c9ca1e3803-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184416    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-lib-modules\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184398    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-sysconfig\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184442    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/035c4d99-4e78-4b8b-a4ab-f0214f882c45-ovnkube-script-lib\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184443    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-kubelet\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184482    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7cba141b-2eef-4463-89e5-87428f1036e1-iptables-alerter-script\") pod \"iptables-alerter-29mkt\" (UID: \"7cba141b-2eef-4463-89e5-87428f1036e1\") " pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184512    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-run-openvswitch\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184535    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-socket-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.185792 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184563    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfkqv\" (UniqueName: \"kubernetes.io/projected/bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa-kube-api-access-bfkqv\") pod \"node-resolver-2zmjn\" (UID: \"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa\") " pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184548    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-lib-modules\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184588    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-socket-dir-parent\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184618    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-daemon-config\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-socket-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-node-log\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184696    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-node-log\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184728    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/035c4d99-4e78-4b8b-a4ab-f0214f882c45-env-overrides\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184748    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-registration-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184810    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-run-k8s-cni-cncf-io\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184852    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-registration-dir\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184855    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-var-lib-cni-bin\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184928    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/035c4d99-4e78-4b8b-a4ab-f0214f882c45-ovn-node-metrics-cert\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184955    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e377a553-4efb-4b16-9679-b8c9ca1e3803-cnibin\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184964    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/035c4d99-4e78-4b8b-a4ab-f0214f882c45-ovnkube-script-lib\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.184983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-var-lib-cni-multus\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185015    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-conf-dir\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185018    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e377a553-4efb-4b16-9679-b8c9ca1e3803-cnibin\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185050    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-run-ovn\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185079    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e377a553-4efb-4b16-9679-b8c9ca1e3803-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185109    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhrrk\" (UniqueName: \"kubernetes.io/projected/e377a553-4efb-4b16-9679-b8c9ca1e3803-kube-api-access-jhrrk\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185122    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-run-ovn\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185137    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185162    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185178    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-run-systemd\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185180    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185196    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-var-lib-openvswitch\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185220    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-cni-dir\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185245    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-run-systemd\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-cnibin\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-var-lib-openvswitch\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185282    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/00dc4e9d-f4e7-41a6-9d88-3411a8df5400-serviceca\") pod \"node-ca-hjsjr\" (UID: \"00dc4e9d-f4e7-41a6-9d88-3411a8df5400\") " pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.186962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185310    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d424p\" (UniqueName: \"kubernetes.io/projected/00dc4e9d-f4e7-41a6-9d88-3411a8df5400-kube-api-access-d424p\") pod \"node-ca-hjsjr\" (UID: \"00dc4e9d-f4e7-41a6-9d88-3411a8df5400\") " pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185476    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e377a553-4efb-4b16-9679-b8c9ca1e3803-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-slash\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-run-netns\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-etc-openvswitch\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185586    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/035c4d99-4e78-4b8b-a4ab-f0214f882c45-ovnkube-config\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185586    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-slash\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znqfk\" (UniqueName: \"kubernetes.io/projected/9c939c52-06f2-4030-846f-c187cbbdaec6-kube-api-access-znqfk\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e377a553-4efb-4b16-9679-b8c9ca1e3803-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185652    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa-hosts-file\") pod \"node-resolver-2zmjn\" (UID: \"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa\") " pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185682    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-sysctl-conf\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-etc-openvswitch\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185704    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-var-lib-kubelet\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/035c4d99-4e78-4b8b-a4ab-f0214f882c45-host-run-netns\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185727    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-etc-selinux\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185787    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9c939c52-06f2-4030-846f-c187cbbdaec6-etc-selinux\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185818    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/00dc4e9d-f4e7-41a6-9d88-3411a8df5400-serviceca\") pod \"node-ca-hjsjr\" (UID: \"00dc4e9d-f4e7-41a6-9d88-3411a8df5400\") " pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.187513 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185892    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-sysctl-conf\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.188126 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185894    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/01b43bc3-9185-4f02-8486-3668fac6aa3f-var-lib-kubelet\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.188126 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.185928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-run-netns\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.188126 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.186127    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/035c4d99-4e78-4b8b-a4ab-f0214f882c45-ovnkube-config\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.188553 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.188524    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/01b43bc3-9185-4f02-8486-3668fac6aa3f-etc-tuned\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.188638 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.188564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/01b43bc3-9185-4f02-8486-3668fac6aa3f-tmp\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.188815 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.188795    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/035c4d99-4e78-4b8b-a4ab-f0214f882c45-ovn-node-metrics-cert\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.204027 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.204006    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4sst\" (UniqueName: \"kubernetes.io/projected/035c4d99-4e78-4b8b-a4ab-f0214f882c45-kube-api-access-f4sst\") pod \"ovnkube-node-zb9z7\" (UID: \"035c4d99-4e78-4b8b-a4ab-f0214f882c45\") " pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.205296 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.205279    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:57:31.205387 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.205299    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:57:31.205387 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.205309    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qpd7v for pod openshift-network-diagnostics/network-check-target-r48dd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:31.205387 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.205359    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v podName:47d2e297-0972-4bad-a4f4-ea260dd620ca nodeName:}" failed. No retries permitted until 2026-09-11 13:57:31.705343865 +0000 UTC m=+2.042919410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qpd7v" (UniqueName: "kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v") pod "network-check-target-r48dd" (UID: "47d2e297-0972-4bad-a4f4-ea260dd620ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:31.207407 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.207383    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdv7f\" (UniqueName: \"kubernetes.io/projected/01b43bc3-9185-4f02-8486-3668fac6aa3f-kube-api-access-mdv7f\") pod \"tuned-6nc5n\" (UID: \"01b43bc3-9185-4f02-8486-3668fac6aa3f\") " pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.207682 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.207658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhrrk\" (UniqueName: \"kubernetes.io/projected/e377a553-4efb-4b16-9679-b8c9ca1e3803-kube-api-access-jhrrk\") pod \"multus-additional-cni-plugins-kf48t\" (UID: \"e377a553-4efb-4b16-9679-b8c9ca1e3803\") " pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.208563 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.208545    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znqfk\" (UniqueName: \"kubernetes.io/projected/9c939c52-06f2-4030-846f-c187cbbdaec6-kube-api-access-znqfk\") pod \"aws-ebs-csi-driver-node-jgjwq\" (UID: \"9c939c52-06f2-4030-846f-c187cbbdaec6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.209243 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.209226    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d424p\" (UniqueName: \"kubernetes.io/projected/00dc4e9d-f4e7-41a6-9d88-3411a8df5400-kube-api-access-d424p\") pod \"node-ca-hjsjr\" (UID: \"00dc4e9d-f4e7-41a6-9d88-3411a8df5400\") " pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.212834 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.212813    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq9kd\" (UniqueName: \"kubernetes.io/projected/7cba141b-2eef-4463-89e5-87428f1036e1-kube-api-access-nq9kd\") pod \"iptables-alerter-29mkt\" (UID: \"7cba141b-2eef-4463-89e5-87428f1036e1\") " pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.284370 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.284349    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:57:31.286728 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-etc-kubernetes\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.286799 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286739    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa-tmp-dir\") pod \"node-resolver-2zmjn\" (UID: \"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa\") " pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.286799 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286755    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9953ef84-7f0d-4d91-8f62-735ee11380b3-kubelet-config\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.286885 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286801    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4ff248cb-b2cd-46e6-865f-c5d26df89969-agent-certs\") pod \"konnectivity-agent-ljzn7\" (UID: \"4ff248cb-b2cd-46e6-865f-c5d26df89969\") " pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:31.286885 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-system-cni-dir\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.286885 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286851    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9953ef84-7f0d-4d91-8f62-735ee11380b3-kubelet-config\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.286885 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-etc-kubernetes\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.286885 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-run-multus-certs\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-system-cni-dir\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9953ef84-7f0d-4d91-8f62-735ee11380b3-dbus\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-run-multus-certs\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.286961    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfkqv\" (UniqueName: \"kubernetes.io/projected/bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa-kube-api-access-bfkqv\") pod \"node-resolver-2zmjn\" (UID: \"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa\") " pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287000    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-socket-dir-parent\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-daemon-config\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287036    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9953ef84-7f0d-4d91-8f62-735ee11380b3-dbus\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-run-k8s-cni-cncf-io\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287077    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa-tmp-dir\") pod \"node-resolver-2zmjn\" (UID: \"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa\") " pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287087    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-var-lib-cni-bin\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287093    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-socket-dir-parent\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287127    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-var-lib-cni-bin\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287129    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-var-lib-cni-multus\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-conf-dir\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287173    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-run-k8s-cni-cncf-io\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.287203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287208    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287215    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-var-lib-cni-multus\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287236    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-cni-dir\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287248    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-conf-dir\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-cnibin\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287292    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa-hosts-file\") pod \"node-resolver-2zmjn\" (UID: \"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa\") " pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.287300    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287318    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-run-netns\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287343    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-cni-dir\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287352    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-cnibin\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.287362    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret podName:9953ef84-7f0d-4d91-8f62-735ee11380b3 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:31.787342764 +0000 UTC m=+2.124918308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret") pod "global-pull-secret-syncer-vz6bg" (UID: "9953ef84-7f0d-4d91-8f62-735ee11380b3") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-var-lib-kubelet\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287414    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa-hosts-file\") pod \"node-resolver-2zmjn\" (UID: \"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa\") " pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287465    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-run-netns\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287500    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-host-var-lib-kubelet\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287427    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlzbm\" (UniqueName: \"kubernetes.io/projected/a8041769-d3ac-4348-9ce8-f77ed826aa8d-kube-api-access-qlzbm\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287539    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8041769-d3ac-4348-9ce8-f77ed826aa8d-cni-binary-copy\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287565    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-hostroot\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288023 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287595    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:31.288684 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287603    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a8041769-d3ac-4348-9ce8-f77ed826aa8d-multus-daemon-config\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288684 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287623    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4ff248cb-b2cd-46e6-865f-c5d26df89969-konnectivity-ca\") pod \"konnectivity-agent-ljzn7\" (UID: \"4ff248cb-b2cd-46e6-865f-c5d26df89969\") " pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:31.288684 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287661    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a8041769-d3ac-4348-9ce8-f77ed826aa8d-hostroot\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.288684 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.287649    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gq5g\" (UniqueName: \"kubernetes.io/projected/73e95838-8a95-4047-b739-0f740d140756-kube-api-access-6gq5g\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:31.288684 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.288180    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:31.288684 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.288231    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs podName:73e95838-8a95-4047-b739-0f740d140756 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:31.788218784 +0000 UTC m=+2.125794329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs") pod "network-metrics-daemon-sjsjk" (UID: "73e95838-8a95-4047-b739-0f740d140756") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:31.288684 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.288176    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4ff248cb-b2cd-46e6-865f-c5d26df89969-konnectivity-ca\") pod \"konnectivity-agent-ljzn7\" (UID: \"4ff248cb-b2cd-46e6-865f-c5d26df89969\") " pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:31.288684 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.288558    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8041769-d3ac-4348-9ce8-f77ed826aa8d-cni-binary-copy\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.289602 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.289584    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4ff248cb-b2cd-46e6-865f-c5d26df89969-agent-certs\") pod \"konnectivity-agent-ljzn7\" (UID: \"4ff248cb-b2cd-46e6-865f-c5d26df89969\") " pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:31.299841 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.299821    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlzbm\" (UniqueName: \"kubernetes.io/projected/a8041769-d3ac-4348-9ce8-f77ed826aa8d-kube-api-access-qlzbm\") pod \"multus-fvqbb\" (UID: \"a8041769-d3ac-4348-9ce8-f77ed826aa8d\") " pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.300419 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.300395    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfkqv\" (UniqueName: \"kubernetes.io/projected/bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa-kube-api-access-bfkqv\") pod \"node-resolver-2zmjn\" (UID: \"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa\") " pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.300635 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.300620    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gq5g\" (UniqueName: \"kubernetes.io/projected/73e95838-8a95-4047-b739-0f740d140756-kube-api-access-6gq5g\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:31.394136 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.393327    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:57:31.403180 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.403115    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:31.419279 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.419215    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kf48t"
Sep 11 13:57:31.436326 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.436307    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hjsjr"
Sep 11 13:57:31.451885 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.451867    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq"
Sep 11 13:57:31.458902 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.458877    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6nc5n"
Sep 11 13:57:31.460568 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.460538    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4517e21fa1b4a954b46d170bcf0a3a.slice/crio-e896f2f1ae6b209807013f94277c92238cc323df9dc2d0428bf1d134f05a5c7b WatchSource:0}: Error finding container e896f2f1ae6b209807013f94277c92238cc323df9dc2d0428bf1d134f05a5c7b: Status 404 returned error can't find the container with id e896f2f1ae6b209807013f94277c92238cc323df9dc2d0428bf1d134f05a5c7b
Sep 11 13:57:31.461022 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.460994    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00dc4e9d_f4e7_41a6_9d88_3411a8df5400.slice/crio-24cd442c2785dc839a9fef65d7685d8a45ea5f2ad776b6dafe6146aa2d0ccbe7 WatchSource:0}: Error finding container 24cd442c2785dc839a9fef65d7685d8a45ea5f2ad776b6dafe6146aa2d0ccbe7: Status 404 returned error can't find the container with id 24cd442c2785dc839a9fef65d7685d8a45ea5f2ad776b6dafe6146aa2d0ccbe7
Sep 11 13:57:31.461958 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.461933    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode377a553_4efb_4b16_9679_b8c9ca1e3803.slice/crio-a515694672a4300b9561973ee7caa925e0e25cb23fa6a235af63101cb534b364 WatchSource:0}: Error finding container a515694672a4300b9561973ee7caa925e0e25cb23fa6a235af63101cb534b364: Status 404 returned error can't find the container with id a515694672a4300b9561973ee7caa925e0e25cb23fa6a235af63101cb534b364
Sep 11 13:57:31.462710 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.462676    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a193ec36d49eb843a04da4a94012319.slice/crio-a8036154541737d799009941683aab9a14af677762354f9c43f6bb60bb38dbac WatchSource:0}: Error finding container a8036154541737d799009941683aab9a14af677762354f9c43f6bb60bb38dbac: Status 404 returned error can't find the container with id a8036154541737d799009941683aab9a14af677762354f9c43f6bb60bb38dbac
Sep 11 13:57:31.466607 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.466588    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:57:31.467665 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.467629    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c939c52_06f2_4030_846f_c187cbbdaec6.slice/crio-dcb2505241f571aea599a064a797b283e49deb8df4d4346c3b88fd44227a6744 WatchSource:0}: Error finding container dcb2505241f571aea599a064a797b283e49deb8df4d4346c3b88fd44227a6744: Status 404 returned error can't find the container with id dcb2505241f571aea599a064a797b283e49deb8df4d4346c3b88fd44227a6744
Sep 11 13:57:31.470308 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.470287    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b43bc3_9185_4f02_8486_3668fac6aa3f.slice/crio-7442eb489f080f26494f6142188e2896f25a25e15ca87641bfa2b8c9f0a2e190 WatchSource:0}: Error finding container 7442eb489f080f26494f6142188e2896f25a25e15ca87641bfa2b8c9f0a2e190: Status 404 returned error can't find the container with id 7442eb489f080f26494f6142188e2896f25a25e15ca87641bfa2b8c9f0a2e190
Sep 11 13:57:31.490825 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.490804    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-29mkt"
Sep 11 13:57:31.499855 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.499831    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cba141b_2eef_4463_89e5_87428f1036e1.slice/crio-ff4ba36f2f2dd282db922325aefb7f4f3e05c4fa354e091427327ff6abd9091c WatchSource:0}: Error finding container ff4ba36f2f2dd282db922325aefb7f4f3e05c4fa354e091427327ff6abd9091c: Status 404 returned error can't find the container with id ff4ba36f2f2dd282db922325aefb7f4f3e05c4fa354e091427327ff6abd9091c
Sep 11 13:57:31.519385 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.519369    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2zmjn"
Sep 11 13:57:31.524355 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.524332    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:31.529251 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.529092    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc389e8a_e4aa_4f96_a52b_2bf944c5bfaa.slice/crio-13d2a2ac9d2367fa24c76ff9730a2a51d50483b5664bbed744fa0957fb2b1bba WatchSource:0}: Error finding container 13d2a2ac9d2367fa24c76ff9730a2a51d50483b5664bbed744fa0957fb2b1bba: Status 404 returned error can't find the container with id 13d2a2ac9d2367fa24c76ff9730a2a51d50483b5664bbed744fa0957fb2b1bba
Sep 11 13:57:31.532858 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.532838    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff248cb_b2cd_46e6_865f_c5d26df89969.slice/crio-092c8efb737479313be751929cc4372a812bd36407f891119766e5ad4e5bda52 WatchSource:0}: Error finding container 092c8efb737479313be751929cc4372a812bd36407f891119766e5ad4e5bda52: Status 404 returned error can't find the container with id 092c8efb737479313be751929cc4372a812bd36407f891119766e5ad4e5bda52
Sep 11 13:57:31.540868 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.540848    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fvqbb"
Sep 11 13:57:31.549981 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.549958    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8041769_d3ac_4348_9ce8_f77ed826aa8d.slice/crio-e41737e9bedc592a6f8f78ac05aa0673ab1ccd61c6303c4e855bb7bbd3837035 WatchSource:0}: Error finding container e41737e9bedc592a6f8f78ac05aa0673ab1ccd61c6303c4e855bb7bbd3837035: Status 404 returned error can't find the container with id e41737e9bedc592a6f8f78ac05aa0673ab1ccd61c6303c4e855bb7bbd3837035
Sep 11 13:57:31.782515 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:57:31.782484    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035c4d99_4e78_4b8b_a4ab_f0214f882c45.slice/crio-e0b670b92bf4ac07ad2cf650ab37dbec9a3f86d6f0ccaf941b272e363456ec54 WatchSource:0}: Error finding container e0b670b92bf4ac07ad2cf650ab37dbec9a3f86d6f0ccaf941b272e363456ec54: Status 404 returned error can't find the container with id e0b670b92bf4ac07ad2cf650ab37dbec9a3f86d6f0ccaf941b272e363456ec54
Sep 11 13:57:31.789841 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.789819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:31.789923 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.789853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:31.789923 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:31.789877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:31.790012 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.789967    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:31.790012 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.789969    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:31.790012 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.789979    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:57:31.790012 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.789997    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:57:31.790012 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.790009    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qpd7v for pod openshift-network-diagnostics/network-check-target-r48dd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:31.790012 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.790015    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs podName:73e95838-8a95-4047-b739-0f740d140756 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:32.7899993 +0000 UTC m=+3.127574840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs") pod "network-metrics-daemon-sjsjk" (UID: "73e95838-8a95-4047-b739-0f740d140756") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:31.790313 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.790031    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret podName:9953ef84-7f0d-4d91-8f62-735ee11380b3 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:32.790022162 +0000 UTC m=+3.127597701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret") pod "global-pull-secret-syncer-vz6bg" (UID: "9953ef84-7f0d-4d91-8f62-735ee11380b3") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:31.790313 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:31.790048    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v podName:47d2e297-0972-4bad-a4f4-ea260dd620ca nodeName:}" failed. No retries permitted until 2026-09-11 13:57:32.790035181 +0000 UTC m=+3.127610720 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpd7v" (UniqueName: "kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v") pod "network-check-target-r48dd" (UID: "47d2e297-0972-4bad-a4f4-ea260dd620ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:32.135008 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.134853    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:52:31 +0000 UTC" deadline="2028-04-26 05:27:27.282942777 +0000 UTC"
Sep 11 13:57:32.135008 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.134902    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14223h29m55.148045519s"
Sep 11 13:57:32.243481 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.243450    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:57:32.285811 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.285751    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" event={"ID":"035c4d99-4e78-4b8b-a4ab-f0214f882c45","Type":"ContainerStarted","Data":"e0b670b92bf4ac07ad2cf650ab37dbec9a3f86d6f0ccaf941b272e363456ec54"}
Sep 11 13:57:32.298249 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.298216    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fvqbb" event={"ID":"a8041769-d3ac-4348-9ce8-f77ed826aa8d","Type":"ContainerStarted","Data":"e41737e9bedc592a6f8f78ac05aa0673ab1ccd61c6303c4e855bb7bbd3837035"}
Sep 11 13:57:32.304165 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.304109    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ljzn7" event={"ID":"4ff248cb-b2cd-46e6-865f-c5d26df89969","Type":"ContainerStarted","Data":"092c8efb737479313be751929cc4372a812bd36407f891119766e5ad4e5bda52"}
Sep 11 13:57:32.313452 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.313400    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-29mkt" event={"ID":"7cba141b-2eef-4463-89e5-87428f1036e1","Type":"ContainerStarted","Data":"ff4ba36f2f2dd282db922325aefb7f4f3e05c4fa354e091427327ff6abd9091c"}
Sep 11 13:57:32.316455 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.316426    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6nc5n" event={"ID":"01b43bc3-9185-4f02-8486-3668fac6aa3f","Type":"ContainerStarted","Data":"7442eb489f080f26494f6142188e2896f25a25e15ca87641bfa2b8c9f0a2e190"}
Sep 11 13:57:32.324906 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.324860    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hjsjr" event={"ID":"00dc4e9d-f4e7-41a6-9d88-3411a8df5400","Type":"ContainerStarted","Data":"24cd442c2785dc839a9fef65d7685d8a45ea5f2ad776b6dafe6146aa2d0ccbe7"}
Sep 11 13:57:32.333890 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.333843    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal" event={"ID":"eb4517e21fa1b4a954b46d170bcf0a3a","Type":"ContainerStarted","Data":"e896f2f1ae6b209807013f94277c92238cc323df9dc2d0428bf1d134f05a5c7b"}
Sep 11 13:57:32.343827 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.343763    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2zmjn" event={"ID":"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa","Type":"ContainerStarted","Data":"13d2a2ac9d2367fa24c76ff9730a2a51d50483b5664bbed744fa0957fb2b1bba"}
Sep 11 13:57:32.347997 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.347968    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq" event={"ID":"9c939c52-06f2-4030-846f-c187cbbdaec6","Type":"ContainerStarted","Data":"dcb2505241f571aea599a064a797b283e49deb8df4d4346c3b88fd44227a6744"}
Sep 11 13:57:32.355053 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.355030    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf48t" event={"ID":"e377a553-4efb-4b16-9679-b8c9ca1e3803","Type":"ContainerStarted","Data":"a515694672a4300b9561973ee7caa925e0e25cb23fa6a235af63101cb534b364"}
Sep 11 13:57:32.359945 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.359893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal" event={"ID":"1a193ec36d49eb843a04da4a94012319","Type":"ContainerStarted","Data":"a8036154541737d799009941683aab9a14af677762354f9c43f6bb60bb38dbac"}
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.798802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.798885    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:32.798917    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:32.798948    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:32.799022    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs podName:73e95838-8a95-4047-b739-0f740d140756 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:34.799002624 +0000 UTC m=+5.136578178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs") pod "network-metrics-daemon-sjsjk" (UID: "73e95838-8a95-4047-b739-0f740d140756") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:32.799036    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:32.799089    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret podName:9953ef84-7f0d-4d91-8f62-735ee11380b3 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:34.799072587 +0000 UTC m=+5.136648139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret") pod "global-pull-secret-syncer-vz6bg" (UID: "9953ef84-7f0d-4d91-8f62-735ee11380b3") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:32.799174    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:32.799191    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:32.799204    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qpd7v for pod openshift-network-diagnostics/network-check-target-r48dd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:32.799279 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:32.799244    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v podName:47d2e297-0972-4bad-a4f4-ea260dd620ca nodeName:}" failed. No retries permitted until 2026-09-11 13:57:34.799230472 +0000 UTC m=+5.136806015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpd7v" (UniqueName: "kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v") pod "network-check-target-r48dd" (UID: "47d2e297-0972-4bad-a4f4-ea260dd620ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:33.136200 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:33.136092    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 13:52:31 +0000 UTC" deadline="2028-06-12 20:21:49.710317086 +0000 UTC"
Sep 11 13:57:33.136200 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:33.136174    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15366h24m16.574181573s"
Sep 11 13:57:33.269001 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:33.268949    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:33.269192 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:33.269118    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:33.269668 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:33.269602    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:33.269770 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:33.269724    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:33.269848 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:33.269797    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:33.269942 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:33.269870    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:34.817369 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:34.817313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:34.817905 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:34.817417    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:34.817905 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:34.817451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:34.817905 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:34.817589    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:34.817905 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:34.817652    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret podName:9953ef84-7f0d-4d91-8f62-735ee11380b3 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:38.817632961 +0000 UTC m=+9.155208519 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret") pod "global-pull-secret-syncer-vz6bg" (UID: "9953ef84-7f0d-4d91-8f62-735ee11380b3") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:34.818151 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:34.818121    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:34.818210 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:34.818175    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs podName:73e95838-8a95-4047-b739-0f740d140756 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:38.818158321 +0000 UTC m=+9.155733877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs") pod "network-metrics-daemon-sjsjk" (UID: "73e95838-8a95-4047-b739-0f740d140756") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:34.818275 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:34.818256    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:57:34.818275 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:34.818270    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:57:34.818385 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:34.818283    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qpd7v for pod openshift-network-diagnostics/network-check-target-r48dd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:34.818385 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:34.818322    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v podName:47d2e297-0972-4bad-a4f4-ea260dd620ca nodeName:}" failed. No retries permitted until 2026-09-11 13:57:38.818310271 +0000 UTC m=+9.155885815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpd7v" (UniqueName: "kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v") pod "network-check-target-r48dd" (UID: "47d2e297-0972-4bad-a4f4-ea260dd620ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:35.269836 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:35.269744    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:35.270009 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:35.269902    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:35.270079 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:35.269901    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:35.270079 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:35.270043    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:35.270195 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:35.270038    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:35.270195 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:35.270147    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:37.269622 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:37.269586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:37.270097 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:37.269729    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:37.270097 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:37.269787    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:37.270097 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:37.269851    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:37.270097 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:37.269895    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:37.270097 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:37.269988    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:38.850188 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:38.850155    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:38.850200    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:38.850239    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:38.850352    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:38.850365    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:38.850384    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:38.850391    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:38.850397    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qpd7v for pod openshift-network-diagnostics/network-check-target-r48dd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:38.850434    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs podName:73e95838-8a95-4047-b739-0f740d140756 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:46.850415285 +0000 UTC m=+17.187990848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs") pod "network-metrics-daemon-sjsjk" (UID: "73e95838-8a95-4047-b739-0f740d140756") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:38.850486    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret podName:9953ef84-7f0d-4d91-8f62-735ee11380b3 nodeName:}" failed. No retries permitted until 2026-09-11 13:57:46.850469202 +0000 UTC m=+17.188044747 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret") pod "global-pull-secret-syncer-vz6bg" (UID: "9953ef84-7f0d-4d91-8f62-735ee11380b3") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:38.850645 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:38.850503    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v podName:47d2e297-0972-4bad-a4f4-ea260dd620ca nodeName:}" failed. No retries permitted until 2026-09-11 13:57:46.850493795 +0000 UTC m=+17.188069340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpd7v" (UniqueName: "kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v") pod "network-check-target-r48dd" (UID: "47d2e297-0972-4bad-a4f4-ea260dd620ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:39.269426 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:39.269325    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:39.269426 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:39.269351    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:39.269426 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:39.269325    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:39.269718 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:39.269438    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:39.269718 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:39.269502    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:39.269718 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:39.269570    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:41.269935 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:41.269908    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:41.270425 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:41.269960    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:41.270425 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:41.270043    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:41.270545 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:41.270454    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:41.270545 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:41.270519    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:41.270681 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:41.270607    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:43.269205 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:43.269177    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:43.269685 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:43.269177    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:43.269685 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:43.269273    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:43.269685 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:43.269342    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:43.269685 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:43.269177    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:43.269685 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:43.269452    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:45.269933 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:45.269896    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:45.269933 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:45.269938    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:45.270485 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:45.269896    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:45.270485 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:45.270029    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:45.270485 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:45.270147    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:45.270485 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:45.270222    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:46.907928 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:46.907893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:46.907928 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:46.907932    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:46.908394 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:46.907970    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:46.908394 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:46.908066    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:57:46.908394 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:46.908073    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:46.908394 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:46.908086    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:57:46.908394 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:46.908098    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qpd7v for pod openshift-network-diagnostics/network-check-target-r48dd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:46.908394 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:46.908128    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:46.908394 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:46.908139    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs podName:73e95838-8a95-4047-b739-0f740d140756 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:02.908120591 +0000 UTC m=+33.245696146 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs") pod "network-metrics-daemon-sjsjk" (UID: "73e95838-8a95-4047-b739-0f740d140756") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:57:46.908394 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:46.908209    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v podName:47d2e297-0972-4bad-a4f4-ea260dd620ca nodeName:}" failed. No retries permitted until 2026-09-11 13:58:02.908190003 +0000 UTC m=+33.245765553 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpd7v" (UniqueName: "kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v") pod "network-check-target-r48dd" (UID: "47d2e297-0972-4bad-a4f4-ea260dd620ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:57:46.908394 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:46.908233    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret podName:9953ef84-7f0d-4d91-8f62-735ee11380b3 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:02.908222809 +0000 UTC m=+33.245798355 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret") pod "global-pull-secret-syncer-vz6bg" (UID: "9953ef84-7f0d-4d91-8f62-735ee11380b3") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:57:47.269784 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:47.269705    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:47.269957 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:47.269811    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:47.269957 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:47.269824    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:47.269957 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:47.269822    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:47.269957 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:47.269908    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:47.270161 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:47.269991    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:49.268962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:49.268916    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:49.269417 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:49.268934    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:49.269417 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:49.269071    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:49.269417 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:49.268916    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:49.269417 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:49.269160    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:49.269417 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:49.269205    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:51.269387 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.269212    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:51.270010 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.269212    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:51.270010 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.269212    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:51.270010 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:51.269517    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:51.270010 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:51.269680    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:51.270010 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:51.269792    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:51.413539 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.413504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6nc5n" event={"ID":"01b43bc3-9185-4f02-8486-3668fac6aa3f","Type":"ContainerStarted","Data":"4b4b99339c5174a19d5018a32bb3bd6ec7863ffb777352f1263fbda6b7115d94"}
Sep 11 13:57:51.419530 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.419501    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal" event={"ID":"1a193ec36d49eb843a04da4a94012319","Type":"ContainerStarted","Data":"d032872adf9215a650bc8e3f93f07d6a3ba9d94d8e88db2a346a008a621c5db7"}
Sep 11 13:57:51.425545 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.425521    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" event={"ID":"035c4d99-4e78-4b8b-a4ab-f0214f882c45","Type":"ContainerStarted","Data":"530e143a4bf700dacc5d771aa19e08ef563aa633799a9a6cbe21a936942bfd7a"}
Sep 11 13:57:51.425650 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.425597    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" event={"ID":"035c4d99-4e78-4b8b-a4ab-f0214f882c45","Type":"ContainerStarted","Data":"0b09452be99fe205bdfa8859e474daa6cf95851338538b8b4040167315ff9532"}
Sep 11 13:57:51.429501 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.429456    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fvqbb" event={"ID":"a8041769-d3ac-4348-9ce8-f77ed826aa8d","Type":"ContainerStarted","Data":"e4a3d0fe8223ff3cade7aa45738be34821b7b951f73ce81c1b82347afc0b2392"}
Sep 11 13:57:51.435541 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.435491    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6nc5n" podStartSLOduration=1.838552463 podStartE2EDuration="21.435475533s" totalImagesPullingTime="19.59692307s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:51.435250151 +0000 UTC m=+21.772825718" watchObservedRunningTime="2026-09-11 13:57:51.435475533 +0000 UTC m=+21.773051094"
Sep 11 13:57:51.470729 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:51.470691    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fvqbb" podStartSLOduration=0.914408672 podStartE2EDuration="20.470674432s" totalImagesPullingTime="19.55626576s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:51.453826046 +0000 UTC m=+21.791401619" watchObservedRunningTime="2026-09-11 13:57:51.470674432 +0000 UTC m=+21.808249991"
Sep 11 13:57:52.433126 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.432915    2564 generic.go:356] "Generic (PLEG): container finished" podID="e377a553-4efb-4b16-9679-b8c9ca1e3803" containerID="bf5b83b970149df37b497971e93b5c5647b4e05bb390b4bb97eb89c1625b85fa" exitCode=0
Sep 11 13:57:52.433821 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.432988    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf48t" event={"ID":"e377a553-4efb-4b16-9679-b8c9ca1e3803","Type":"ContainerDied","Data":"bf5b83b970149df37b497971e93b5c5647b4e05bb390b4bb97eb89c1625b85fa"}
Sep 11 13:57:52.440054 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.440026    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" event={"ID":"035c4d99-4e78-4b8b-a4ab-f0214f882c45","Type":"ContainerStarted","Data":"6a8c106156306f3c282968d6b5378c6a560b8851c0f43891f21371e0da8f4e13"}
Sep 11 13:57:52.440158 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.440138    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" event={"ID":"035c4d99-4e78-4b8b-a4ab-f0214f882c45","Type":"ContainerStarted","Data":"f6ff7f8d4bf58ae3ed5cb194960e452ccb20410e0f6dedf782c3dc4742b69dda"}
Sep 11 13:57:52.440207 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.440191    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" event={"ID":"035c4d99-4e78-4b8b-a4ab-f0214f882c45","Type":"ContainerStarted","Data":"a509e3fad1c3a8888f8df59faa075e032b0feee3d2fea560e456fdeb82452186"}
Sep 11 13:57:52.440298 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.440239    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" event={"ID":"035c4d99-4e78-4b8b-a4ab-f0214f882c45","Type":"ContainerStarted","Data":"f09c845ba56d402786b9a7618ff7fd3a0a5e7de639d47ba64e254745929702a4"}
Sep 11 13:57:52.441679 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.441655    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ljzn7" event={"ID":"4ff248cb-b2cd-46e6-865f-c5d26df89969","Type":"ContainerStarted","Data":"4b67a462d5965cf6c5ab81cfee433df8e7a0168c6a69313e5c3c772b238ad464"}
Sep 11 13:57:52.443376 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.443345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-29mkt" event={"ID":"7cba141b-2eef-4463-89e5-87428f1036e1","Type":"ContainerStarted","Data":"4c400b915af5229c1e14ad22dc29aa5dd601a5dfcd4f5a1567579950a93988d5"}
Sep 11 13:57:52.444919 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.444896    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hjsjr" event={"ID":"00dc4e9d-f4e7-41a6-9d88-3411a8df5400","Type":"ContainerStarted","Data":"a5dcdc383dd86255f54bd3acbe07608dbeae47b8ca72427b7529a5048e0e3e97"}
Sep 11 13:57:52.446486 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.446465    2564 generic.go:356] "Generic (PLEG): container finished" podID="eb4517e21fa1b4a954b46d170bcf0a3a" containerID="b0042a79f698105d68451cf64bdf709cc49890f25c008048f6ef4b8ed01c6c5e" exitCode=0
Sep 11 13:57:52.446593 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.446495    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal" event={"ID":"eb4517e21fa1b4a954b46d170bcf0a3a","Type":"ContainerDied","Data":"b0042a79f698105d68451cf64bdf709cc49890f25c008048f6ef4b8ed01c6c5e"}
Sep 11 13:57:52.448131 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.448109    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2zmjn" event={"ID":"bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa","Type":"ContainerStarted","Data":"48fcb00ccd1c4ef2eaf41662239dc49d5c39a22d3377bfdffd02534dcd28a139"}
Sep 11 13:57:52.449726 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.449674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq" event={"ID":"9c939c52-06f2-4030-846f-c187cbbdaec6","Type":"ContainerStarted","Data":"bdf9de73277bd3a9d8bb0277fbadf67cbb407257fca69dbdb7435b55fd91cde5"}
Sep 11 13:57:52.463266 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.463226    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-195.ec2.internal" podStartSLOduration=21.463215068 podStartE2EDuration="21.463215068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:51.477447116 +0000 UTC m=+21.815022678" watchObservedRunningTime="2026-09-11 13:57:52.463215068 +0000 UTC m=+22.800790646"
Sep 11 13:57:52.506517 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.506471    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ljzn7" podStartSLOduration=1.972222424 podStartE2EDuration="21.506457566s" totalImagesPullingTime="19.534235142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:52.484717468 +0000 UTC m=+22.822293030" watchObservedRunningTime="2026-09-11 13:57:52.506457566 +0000 UTC m=+22.844033125"
Sep 11 13:57:52.529285 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.529250    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-29mkt" podStartSLOduration=2.9694853549999998 podStartE2EDuration="22.529235853s" totalImagesPullingTime="19.559750498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:52.528853772 +0000 UTC m=+22.866429333" watchObservedRunningTime="2026-09-11 13:57:52.529235853 +0000 UTC m=+22.866811416"
Sep 11 13:57:52.558052 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.558004    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2zmjn" podStartSLOduration=3.019626757 podStartE2EDuration="22.557988767s" totalImagesPullingTime="19.53836201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:52.557907601 +0000 UTC m=+22.895483160" watchObservedRunningTime="2026-09-11 13:57:52.557988767 +0000 UTC m=+22.895564326"
Sep 11 13:57:52.577953 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:52.577912    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hjsjr" podStartSLOduration=2.9833045179999997 podStartE2EDuration="22.577900215s" totalImagesPullingTime="19.594595697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:52.577855512 +0000 UTC m=+22.915431071" watchObservedRunningTime="2026-09-11 13:57:52.577900215 +0000 UTC m=+22.915475776"
Sep 11 13:57:53.034266 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.034116    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 13:57:53.154058 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.153965    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T13:57:53.034261394Z","UUID":"e5bc5e92-49bc-495a-9175-becd2e7b9c51","Handler":null,"Name":"","Endpoint":""}
Sep 11 13:57:53.155469 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.155439    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 13:57:53.155469 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.155470    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 13:57:53.269740 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.269619    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:53.269740 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.269656    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:53.269984 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.269748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:53.269984 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:53.269753    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:53.269984 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:53.269847    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:53.269984 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:53.269920    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:53.454546 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.454505    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq" event={"ID":"9c939c52-06f2-4030-846f-c187cbbdaec6","Type":"ContainerStarted","Data":"2ec124641f2bc392669f5dd0b7ab3e4266cba24484aabde1c9ac9a29e49afa2e"}
Sep 11 13:57:53.456621 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.456590    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal" event={"ID":"eb4517e21fa1b4a954b46d170bcf0a3a","Type":"ContainerStarted","Data":"b01bd18c099b6fab777837b8276940a06ce758692acaf677b75433f0f0bef9b5"}
Sep 11 13:57:53.479592 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:53.479544    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-195.ec2.internal" podStartSLOduration=22.479525664 podStartE2EDuration="22.479525664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:53.477793791 +0000 UTC m=+23.815369352" watchObservedRunningTime="2026-09-11 13:57:53.479525664 +0000 UTC m=+23.817101225"
Sep 11 13:57:54.462389 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:54.462358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" event={"ID":"035c4d99-4e78-4b8b-a4ab-f0214f882c45","Type":"ContainerStarted","Data":"8b4d9675ad72c4be6f9f98f8f1bfe6717f980fe1e2bb764ba56b479348446e1b"}
Sep 11 13:57:55.268977 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:55.268948    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:55.268977 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:55.268975    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:55.269181 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:55.269056    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:55.269181 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:55.269100    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:55.269246 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:55.269191    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:55.269281 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:55.269259    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:55.467586 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:55.467557    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq" event={"ID":"9c939c52-06f2-4030-846f-c187cbbdaec6","Type":"ContainerStarted","Data":"555cf5ed95092d211a5e81f4d96446b4ad243670f0fb7e245e16158c4124e9c3"}
Sep 11 13:57:55.491133 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:55.491087    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgjwq" podStartSLOduration=3.3331217349999998 podStartE2EDuration="25.49107296s" totalImagesPullingTime="22.157951225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:55.490921776 +0000 UTC m=+25.828497335" watchObservedRunningTime="2026-09-11 13:57:55.49107296 +0000 UTC m=+25.828648519"
Sep 11 13:57:56.221845 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:56.221813    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:56.222432 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:56.222407    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:56.470131 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:56.470046    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:56.470520 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:56.470416    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ljzn7"
Sep 11 13:57:57.269195 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:57.269019    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:57.269302 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:57.269014    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:57.269365 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:57.269315    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:57.269365 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:57.269014    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:57.269461 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:57.269341    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:57:57.269508 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:57.269452    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:57.473391 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:57.473321    2564 generic.go:356] "Generic (PLEG): container finished" podID="e377a553-4efb-4b16-9679-b8c9ca1e3803" containerID="a63967d420fc688e7f0c5a0e43b126c8916194e547577a70a367e4f312e790d6" exitCode=0
Sep 11 13:57:57.473768 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:57.473399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf48t" event={"ID":"e377a553-4efb-4b16-9679-b8c9ca1e3803","Type":"ContainerDied","Data":"a63967d420fc688e7f0c5a0e43b126c8916194e547577a70a367e4f312e790d6"}
Sep 11 13:57:57.477438 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:57.477415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" event={"ID":"035c4d99-4e78-4b8b-a4ab-f0214f882c45","Type":"ContainerStarted","Data":"8c713d2efb4aa48d9b9b81f1ac8a89befa6e038610b021d2e197e794928e534a"}
Sep 11 13:57:57.572929 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:57.572887    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7" podStartSLOduration=8.248551422 podStartE2EDuration="27.572873923s" totalImagesPullingTime="19.324322501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:57:57.572840465 +0000 UTC m=+27.910416039" watchObservedRunningTime="2026-09-11 13:57:57.572873923 +0000 UTC m=+27.910449509"
Sep 11 13:57:58.481710 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.481677    2564 generic.go:356] "Generic (PLEG): container finished" podID="e377a553-4efb-4b16-9679-b8c9ca1e3803" containerID="9faa4fd4e94fb095a2e9b715357c40dd01a270d65f7148d5a3e28b3c29272d54" exitCode=0
Sep 11 13:57:58.482297 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.481756    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf48t" event={"ID":"e377a553-4efb-4b16-9679-b8c9ca1e3803","Type":"ContainerDied","Data":"9faa4fd4e94fb095a2e9b715357c40dd01a270d65f7148d5a3e28b3c29272d54"}
Sep 11 13:57:58.482472 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.482453    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:58.482650 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.482537    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:58.482650 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.482589    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:58.498044 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.498018    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:58.498190 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.498177    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:57:58.931074 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.930998    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sjsjk"]
Sep 11 13:57:58.931222 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.931176    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:57:58.931325 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:58.931295    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:57:58.937865 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.937845    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vz6bg"]
Sep 11 13:57:58.937987 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.937924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:57:58.938048 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:58.937997    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:57:58.945327 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.945304    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r48dd"]
Sep 11 13:57:58.945435 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:57:58.945384    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:57:58.945486 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:57:58.945453    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:58:00.270848 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:00.270814    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:58:00.271339 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:00.270928    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:58:00.271339 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:00.270982    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:58:00.271339 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:00.271106    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:58:00.489530 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:00.489493    2564 generic.go:356] "Generic (PLEG): container finished" podID="e377a553-4efb-4b16-9679-b8c9ca1e3803" containerID="589fefff8a7b978024286796c89f604e0e27de44081710bea5caa36241e95bcd" exitCode=0
Sep 11 13:58:00.489743 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:00.489568    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf48t" event={"ID":"e377a553-4efb-4b16-9679-b8c9ca1e3803","Type":"ContainerDied","Data":"589fefff8a7b978024286796c89f604e0e27de44081710bea5caa36241e95bcd"}
Sep 11 13:58:01.269426 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:01.269188    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:58:01.269570 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:01.269544    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:58:02.269056 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:02.269018    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:58:02.269518 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.269155    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:58:02.269518 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:02.269211    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:58:02.269518 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.269332    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:58:02.923694 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:02.923656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:58:02.923856 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:02.923710    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:58:02.923856 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:02.923799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:58:02.923856 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.923807    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:58:02.923976 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.923869    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs podName:73e95838-8a95-4047-b739-0f740d140756 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:34.92385423 +0000 UTC m=+65.261429772 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs") pod "network-metrics-daemon-sjsjk" (UID: "73e95838-8a95-4047-b739-0f740d140756") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:58:02.923976 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.923879    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:58:02.923976 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.923920    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret podName:9953ef84-7f0d-4d91-8f62-735ee11380b3 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:34.923909067 +0000 UTC m=+65.261484605 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret") pod "global-pull-secret-syncer-vz6bg" (UID: "9953ef84-7f0d-4d91-8f62-735ee11380b3") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:58:02.923976 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.923932    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:58:02.923976 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.923949    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:58:02.923976 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.923959    2564 projected.go:196] Error preparing data for projected volume kube-api-access-qpd7v for pod openshift-network-diagnostics/network-check-target-r48dd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:58:02.924189 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:02.923995    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v podName:47d2e297-0972-4bad-a4f4-ea260dd620ca nodeName:}" failed. No retries permitted until 2026-09-11 13:58:34.923985093 +0000 UTC m=+65.261560636 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qpd7v" (UniqueName: "kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v") pod "network-check-target-r48dd" (UID: "47d2e297-0972-4bad-a4f4-ea260dd620ca") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:58:03.268983 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:03.268895    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:58:03.269124 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:03.269029    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vz6bg" podUID="9953ef84-7f0d-4d91-8f62-735ee11380b3"
Sep 11 13:58:04.269343 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:04.269313    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:58:04.269763 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:04.269325    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:58:04.269763 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:04.269441    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r48dd" podUID="47d2e297-0972-4bad-a4f4-ea260dd620ca"
Sep 11 13:58:04.269763 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:04.269555    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sjsjk" podUID="73e95838-8a95-4047-b739-0f740d140756"
Sep 11 13:58:05.015532 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.015454    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-195.ec2.internal" event="NodeReady"
Sep 11 13:58:05.015705 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.015595    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 13:58:05.057545 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.057512    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d89d5975b-jzjxc"]
Sep 11 13:58:05.080206 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.080182    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k6t77"]
Sep 11 13:58:05.080360 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.080321    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.083151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.083117    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 13:58:05.083151 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.083138    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 13:58:05.083580 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.083561    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 13:58:05.083812 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.083797    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kzr4h\""
Sep 11 13:58:05.090077 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.090056    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 13:58:05.092011 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.091991    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d89d5975b-jzjxc"]
Sep 11 13:58:05.092113 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.092026    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k6t77"]
Sep 11 13:58:05.092113 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.092107    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k6t77"
Sep 11 13:58:05.094337 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.094318    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 13:58:05.095193 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.095175    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 13:58:05.095293 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.095178    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 13:58:05.095293 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.095218    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rh6t5\""
Sep 11 13:58:05.140752 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.140720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.140896 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.140766    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-registry-certificates\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.140896 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.140795    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-installation-pull-secrets\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.140896 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.140823    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-bound-sa-token\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.140896 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.140869    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsx8d\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-kube-api-access-lsx8d\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.141040 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.140907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-image-registry-private-configuration\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.141040 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.140937    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63052ae9-d645-4427-9397-527caeb4823f-ca-trust-extracted\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.141040 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.140960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-trusted-ca\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.169973 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.169948    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p7bff"]
Sep 11 13:58:05.194618 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.194591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p7bff"]
Sep 11 13:58:05.194746 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.194712    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.197143 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.197119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 13:58:05.197143 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.197143    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tbm52\""
Sep 11 13:58:05.197325 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.197225    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 13:58:05.242262 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-trusted-ca\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.242395 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242283    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck6l4\" (UniqueName: \"kubernetes.io/projected/2af3aa4c-d665-483e-874c-0d085d6d1da9-kube-api-access-ck6l4\") pod \"ingress-canary-k6t77\" (UID: \"2af3aa4c-d665-483e-874c-0d085d6d1da9\") " pod="openshift-ingress-canary/ingress-canary-k6t77"
Sep 11 13:58:05.242395 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.242395 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242386    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-registry-certificates\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.242557 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242433    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-installation-pull-secrets\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.242557 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242473    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-bound-sa-token\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.242692 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsx8d\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-kube-api-access-lsx8d\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.242692 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.242517    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:58:05.242692 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242649    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2af3aa4c-d665-483e-874c-0d085d6d1da9-cert\") pod \"ingress-canary-k6t77\" (UID: \"2af3aa4c-d665-483e-874c-0d085d6d1da9\") " pod="openshift-ingress-canary/ingress-canary-k6t77"
Sep 11 13:58:05.242692 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.242669    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d89d5975b-jzjxc: secret "image-registry-tls" not found
Sep 11 13:58:05.242882 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-image-registry-private-configuration\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.242882 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.242743    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls podName:63052ae9-d645-4427-9397-527caeb4823f nodeName:}" failed. No retries permitted until 2026-09-11 13:58:05.742717222 +0000 UTC m=+36.080292784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls") pod "image-registry-d89d5975b-jzjxc" (UID: "63052ae9-d645-4427-9397-527caeb4823f") : secret "image-registry-tls" not found
Sep 11 13:58:05.242882 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.242791    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63052ae9-d645-4427-9397-527caeb4823f-ca-trust-extracted\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.243223 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.243202    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-registry-certificates\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.243450 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.243410    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63052ae9-d645-4427-9397-527caeb4823f-ca-trust-extracted\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.243613 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.243594    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-trusted-ca\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.247351 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.247334    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-installation-pull-secrets\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.247453 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.247358    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-image-registry-private-configuration\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.255110 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.255090    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-bound-sa-token\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.255338 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.255315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsx8d\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-kube-api-access-lsx8d\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.269095 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.269043    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:58:05.271597 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.271573    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:58:05.328380 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.328342    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ljzn7_4ff248cb-b2cd-46e6-865f-c5d26df89969/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:05.343148 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.343121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a7405c-5ea0-49dc-a61b-c3397ec12438-tmp-dir\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.343254 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.343163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2af3aa4c-d665-483e-874c-0d085d6d1da9-cert\") pod \"ingress-canary-k6t77\" (UID: \"2af3aa4c-d665-483e-874c-0d085d6d1da9\") " pod="openshift-ingress-canary/ingress-canary-k6t77"
Sep 11 13:58:05.343254 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.343192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6a7405c-5ea0-49dc-a61b-c3397ec12438-config-volume\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.343254 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.343240    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck6l4\" (UniqueName: \"kubernetes.io/projected/2af3aa4c-d665-483e-874c-0d085d6d1da9-kube-api-access-ck6l4\") pod \"ingress-canary-k6t77\" (UID: \"2af3aa4c-d665-483e-874c-0d085d6d1da9\") " pod="openshift-ingress-canary/ingress-canary-k6t77"
Sep 11 13:58:05.343410 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.343310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.343410 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.343346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcdj9\" (UniqueName: \"kubernetes.io/projected/d6a7405c-5ea0-49dc-a61b-c3397ec12438-kube-api-access-dcdj9\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.345741 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.345721    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2af3aa4c-d665-483e-874c-0d085d6d1da9-cert\") pod \"ingress-canary-k6t77\" (UID: \"2af3aa4c-d665-483e-874c-0d085d6d1da9\") " pod="openshift-ingress-canary/ingress-canary-k6t77"
Sep 11 13:58:05.356031 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.356010    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck6l4\" (UniqueName: \"kubernetes.io/projected/2af3aa4c-d665-483e-874c-0d085d6d1da9-kube-api-access-ck6l4\") pod \"ingress-canary-k6t77\" (UID: \"2af3aa4c-d665-483e-874c-0d085d6d1da9\") " pod="openshift-ingress-canary/ingress-canary-k6t77"
Sep 11 13:58:05.401519 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.401492    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k6t77"
Sep 11 13:58:05.421856 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.421807    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ljzn7_4ff248cb-b2cd-46e6-865f-c5d26df89969/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:05.444672 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.444647    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a7405c-5ea0-49dc-a61b-c3397ec12438-tmp-dir\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.444791 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.444689    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6a7405c-5ea0-49dc-a61b-c3397ec12438-config-volume\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.444791 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.444773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.444910 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.444796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcdj9\" (UniqueName: \"kubernetes.io/projected/d6a7405c-5ea0-49dc-a61b-c3397ec12438-kube-api-access-dcdj9\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.444967 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.444912    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:58:05.445018 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.444974    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls podName:d6a7405c-5ea0-49dc-a61b-c3397ec12438 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:05.944956719 +0000 UTC m=+36.282532272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls") pod "dns-default-p7bff" (UID: "d6a7405c-5ea0-49dc-a61b-c3397ec12438") : secret "dns-default-metrics-tls" not found
Sep 11 13:58:05.445076 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.445023    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6a7405c-5ea0-49dc-a61b-c3397ec12438-tmp-dir\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.445278 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.445259    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d6a7405c-5ea0-49dc-a61b-c3397ec12438-config-volume\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.460461 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.460428    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcdj9\" (UniqueName: \"kubernetes.io/projected/d6a7405c-5ea0-49dc-a61b-c3397ec12438-kube-api-access-dcdj9\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.747544 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.747505    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:05.747724 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.747702    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:58:05.747789 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.747726    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d89d5975b-jzjxc: secret "image-registry-tls" not found
Sep 11 13:58:05.747840 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.747799    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls podName:63052ae9-d645-4427-9397-527caeb4823f nodeName:}" failed. No retries permitted until 2026-09-11 13:58:06.747777396 +0000 UTC m=+37.085352937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls") pod "image-registry-d89d5975b-jzjxc" (UID: "63052ae9-d645-4427-9397-527caeb4823f") : secret "image-registry-tls" not found
Sep 11 13:58:05.852026 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.851960    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k6t77"]
Sep 11 13:58:05.855201 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:58:05.855176    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af3aa4c_d665_483e_874c_0d085d6d1da9.slice/crio-4b19dca8f025c55fd33b252bc68ab5befac19abb6e3d163c218470be8cd8f8dd WatchSource:0}: Error finding container 4b19dca8f025c55fd33b252bc68ab5befac19abb6e3d163c218470be8cd8f8dd: Status 404 returned error can't find the container with id 4b19dca8f025c55fd33b252bc68ab5befac19abb6e3d163c218470be8cd8f8dd
Sep 11 13:58:05.949746 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:05.949584    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:05.949746 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.949734    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:58:05.949967 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:05.949815    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls podName:d6a7405c-5ea0-49dc-a61b-c3397ec12438 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:06.949793816 +0000 UTC m=+37.287369364 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls") pod "dns-default-p7bff" (UID: "d6a7405c-5ea0-49dc-a61b-c3397ec12438") : secret "dns-default-metrics-tls" not found
Sep 11 13:58:06.269839 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.269808    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:58:06.270029 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.270001    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:58:06.272686 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.272664    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:58:06.273061 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.272670    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:58:06.273061 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.272698    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:58:06.273061 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.272698    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fgrdt\""
Sep 11 13:58:06.273061 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.272838    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qh858\""
Sep 11 13:58:06.506203 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.506118    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k6t77" event={"ID":"2af3aa4c-d665-483e-874c-0d085d6d1da9","Type":"ContainerStarted","Data":"4b19dca8f025c55fd33b252bc68ab5befac19abb6e3d163c218470be8cd8f8dd"}
Sep 11 13:58:06.509641 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.509594    2564 generic.go:356] "Generic (PLEG): container finished" podID="e377a553-4efb-4b16-9679-b8c9ca1e3803" containerID="5d20f2cb56a6f4ec7c0a819ad8696493417ecd1acefd8a37c9d24f3cbc4d9171" exitCode=0
Sep 11 13:58:06.509773 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.509653    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf48t" event={"ID":"e377a553-4efb-4b16-9679-b8c9ca1e3803","Type":"ContainerDied","Data":"5d20f2cb56a6f4ec7c0a819ad8696493417ecd1acefd8a37c9d24f3cbc4d9171"}
Sep 11 13:58:06.755489 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.755451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:06.755665 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:06.755644    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:58:06.755736 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:06.755668    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d89d5975b-jzjxc: secret "image-registry-tls" not found
Sep 11 13:58:06.755736 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:06.755724    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls podName:63052ae9-d645-4427-9397-527caeb4823f nodeName:}" failed. No retries permitted until 2026-09-11 13:58:08.755708863 +0000 UTC m=+39.093284405 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls") pod "image-registry-d89d5975b-jzjxc" (UID: "63052ae9-d645-4427-9397-527caeb4823f") : secret "image-registry-tls" not found
Sep 11 13:58:06.957095 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:06.957060    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:06.957273 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:06.957238    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:58:06.957343 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:06.957320    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls podName:d6a7405c-5ea0-49dc-a61b-c3397ec12438 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:08.957297339 +0000 UTC m=+39.294872904 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls") pod "dns-default-p7bff" (UID: "d6a7405c-5ea0-49dc-a61b-c3397ec12438") : secret "dns-default-metrics-tls" not found
Sep 11 13:58:07.009013 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:07.008975    2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ljzn7_4ff248cb-b2cd-46e6-865f-c5d26df89969/konnectivity-agent/0.log" limit=1024
Sep 11 13:58:07.516374 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:07.516327    2564 generic.go:356] "Generic (PLEG): container finished" podID="e377a553-4efb-4b16-9679-b8c9ca1e3803" containerID="7f95ea1141361bdd5d29e2eb74c0ab197c8760ff32afb84d10670a6592cf3950" exitCode=0
Sep 11 13:58:07.517079 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:07.516401    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf48t" event={"ID":"e377a553-4efb-4b16-9679-b8c9ca1e3803","Type":"ContainerDied","Data":"7f95ea1141361bdd5d29e2eb74c0ab197c8760ff32afb84d10670a6592cf3950"}
Sep 11 13:58:08.521313 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:08.521237    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kf48t" event={"ID":"e377a553-4efb-4b16-9679-b8c9ca1e3803","Type":"ContainerStarted","Data":"3bab8944efdd4a3e88ddd6e22d6809a2d05a382c5d011b171da7ee397ce225a2"}
Sep 11 13:58:08.546105 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:08.546066    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kf48t" podStartSLOduration=6.82607372 podStartE2EDuration="38.546052825s" totalImagesPullingTime="31.719979105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:30 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:08.543416825 +0000 UTC m=+38.880992385" watchObservedRunningTime="2026-09-11 13:58:08.546052825 +0000 UTC m=+38.883628384"
Sep 11 13:58:08.771756 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:08.771530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:08.771756 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:08.771691    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:58:08.771756 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:08.771753    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d89d5975b-jzjxc: secret "image-registry-tls" not found
Sep 11 13:58:08.772006 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:08.771796    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls podName:63052ae9-d645-4427-9397-527caeb4823f nodeName:}" failed. No retries permitted until 2026-09-11 13:58:12.771783419 +0000 UTC m=+43.109358969 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls") pod "image-registry-d89d5975b-jzjxc" (UID: "63052ae9-d645-4427-9397-527caeb4823f") : secret "image-registry-tls" not found
Sep 11 13:58:08.973462 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:08.973430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:08.973680 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:08.973561    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:58:08.973680 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:08.973614    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls podName:d6a7405c-5ea0-49dc-a61b-c3397ec12438 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:12.973601194 +0000 UTC m=+43.311176732 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls") pod "dns-default-p7bff" (UID: "d6a7405c-5ea0-49dc-a61b-c3397ec12438") : secret "dns-default-metrics-tls" not found
Sep 11 13:58:09.525031 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:09.525000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k6t77" event={"ID":"2af3aa4c-d665-483e-874c-0d085d6d1da9","Type":"ContainerStarted","Data":"648afa56e49c2fdec10677c65ea239f927e77019326ab1ff8d7c03f5f8370c15"}
Sep 11 13:58:09.544840 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:09.544786    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k6t77" podStartSLOduration=1.765448243 podStartE2EDuration="4.544770698s" totalImagesPullingTime="2.779322455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:09.544440342 +0000 UTC m=+39.882015898" watchObservedRunningTime="2026-09-11 13:58:09.544770698 +0000 UTC m=+39.882346259"
Sep 11 13:58:12.797713 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:12.797677    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:12.798099 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:12.797831    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:58:12.798099 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:12.797850    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d89d5975b-jzjxc: secret "image-registry-tls" not found
Sep 11 13:58:12.798099 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:12.797908    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls podName:63052ae9-d645-4427-9397-527caeb4823f nodeName:}" failed. No retries permitted until 2026-09-11 13:58:20.797892441 +0000 UTC m=+51.135467979 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls") pod "image-registry-d89d5975b-jzjxc" (UID: "63052ae9-d645-4427-9397-527caeb4823f") : secret "image-registry-tls" not found
Sep 11 13:58:12.999173 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:12.999140    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:12.999328 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:12.999297    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:58:12.999367 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:12.999359    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls podName:d6a7405c-5ea0-49dc-a61b-c3397ec12438 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:20.999345426 +0000 UTC m=+51.336920967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls") pod "dns-default-p7bff" (UID: "d6a7405c-5ea0-49dc-a61b-c3397ec12438") : secret "dns-default-metrics-tls" not found
Sep 11 13:58:20.857022 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:20.856981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:20.857451 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:20.857129    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:58:20.857451 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:20.857149    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d89d5975b-jzjxc: secret "image-registry-tls" not found
Sep 11 13:58:20.857451 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:20.857210    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls podName:63052ae9-d645-4427-9397-527caeb4823f nodeName:}" failed. No retries permitted until 2026-09-11 13:58:36.857196025 +0000 UTC m=+67.194771563 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls") pod "image-registry-d89d5975b-jzjxc" (UID: "63052ae9-d645-4427-9397-527caeb4823f") : secret "image-registry-tls" not found
Sep 11 13:58:21.059152 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:21.059116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:21.059310 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:21.059227    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:58:21.059310 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:21.059287    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls podName:d6a7405c-5ea0-49dc-a61b-c3397ec12438 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:37.059274059 +0000 UTC m=+67.396849609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls") pod "dns-default-p7bff" (UID: "d6a7405c-5ea0-49dc-a61b-c3397ec12438") : secret "dns-default-metrics-tls" not found
Sep 11 13:58:30.499621 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:30.499594    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zb9z7"
Sep 11 13:58:34.957241 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.957199    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:58:34.957241 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.957243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:58:34.957884 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.957288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:58:34.959862 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.959839    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:58:34.959862 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.959852    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:58:34.960022 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.959913    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:58:34.967974 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:34.967955    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 13:58:34.968044 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:34.968022    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs podName:73e95838-8a95-4047-b739-0f740d140756 nodeName:}" failed. No retries permitted until 2026-09-11 13:59:38.968007498 +0000 UTC m=+129.305583040 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs") pod "network-metrics-daemon-sjsjk" (UID: "73e95838-8a95-4047-b739-0f740d140756") : secret "metrics-daemon-secret" not found
Sep 11 13:58:34.970020 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.970004    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:58:34.971079 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.971058    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9953ef84-7f0d-4d91-8f62-735ee11380b3-original-pull-secret\") pod \"global-pull-secret-syncer-vz6bg\" (UID: \"9953ef84-7f0d-4d91-8f62-735ee11380b3\") " pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:58:34.977983 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.977966    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vz6bg"
Sep 11 13:58:34.981290 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:34.981270    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpd7v\" (UniqueName: \"kubernetes.io/projected/47d2e297-0972-4bad-a4f4-ea260dd620ca-kube-api-access-qpd7v\") pod \"network-check-target-r48dd\" (UID: \"47d2e297-0972-4bad-a4f4-ea260dd620ca\") " pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:58:35.087607 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:35.087577    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fgrdt\""
Sep 11 13:58:35.095535 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:35.095518    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:58:35.251512 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:35.249684    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vz6bg"]
Sep 11 13:58:35.254843 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:58:35.254817    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9953ef84_7f0d_4d91_8f62_735ee11380b3.slice/crio-86a17027081082c90987fe6fa19d5d2101723de584d2f052b753cd8c27de290b WatchSource:0}: Error finding container 86a17027081082c90987fe6fa19d5d2101723de584d2f052b753cd8c27de290b: Status 404 returned error can't find the container with id 86a17027081082c90987fe6fa19d5d2101723de584d2f052b753cd8c27de290b
Sep 11 13:58:35.364031 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:35.364007    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r48dd"]
Sep 11 13:58:35.366646 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:58:35.366603    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d2e297_0972_4bad_a4f4_ea260dd620ca.slice/crio-2b6b1b22b3edb100dd626989b4a2ccc156be64b80ee22c3b120adba076a91266 WatchSource:0}: Error finding container 2b6b1b22b3edb100dd626989b4a2ccc156be64b80ee22c3b120adba076a91266: Status 404 returned error can't find the container with id 2b6b1b22b3edb100dd626989b4a2ccc156be64b80ee22c3b120adba076a91266
Sep 11 13:58:35.583810 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:35.583775    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r48dd" event={"ID":"47d2e297-0972-4bad-a4f4-ea260dd620ca","Type":"ContainerStarted","Data":"2b6b1b22b3edb100dd626989b4a2ccc156be64b80ee22c3b120adba076a91266"}
Sep 11 13:58:35.584775 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:35.584754    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vz6bg" event={"ID":"9953ef84-7f0d-4d91-8f62-735ee11380b3","Type":"ContainerStarted","Data":"86a17027081082c90987fe6fa19d5d2101723de584d2f052b753cd8c27de290b"}
Sep 11 13:58:36.871992 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:36.871947    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:58:36.872583 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:36.872129    2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 13:58:36.872583 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:36.872152    2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d89d5975b-jzjxc: secret "image-registry-tls" not found
Sep 11 13:58:36.872583 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:36.872262    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls podName:63052ae9-d645-4427-9397-527caeb4823f nodeName:}" failed. No retries permitted until 2026-09-11 13:59:08.8722409 +0000 UTC m=+99.209816453 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls") pod "image-registry-d89d5975b-jzjxc" (UID: "63052ae9-d645-4427-9397-527caeb4823f") : secret "image-registry-tls" not found
Sep 11 13:58:37.073663 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:37.073611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:58:37.073914 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:37.073846    2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:58:37.073996 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:37.073916    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls podName:d6a7405c-5ea0-49dc-a61b-c3397ec12438 nodeName:}" failed. No retries permitted until 2026-09-11 13:59:09.073898234 +0000 UTC m=+99.411473786 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls") pod "dns-default-p7bff" (UID: "d6a7405c-5ea0-49dc-a61b-c3397ec12438") : secret "dns-default-metrics-tls" not found
Sep 11 13:58:40.598771 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:40.598730    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vz6bg" event={"ID":"9953ef84-7f0d-4d91-8f62-735ee11380b3","Type":"ContainerStarted","Data":"b0e465e6b7fa9852663444f39c6131a06c162327e10d4b1eb756f247c5f83a88"}
Sep 11 13:58:40.600231 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:40.600208    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r48dd" event={"ID":"47d2e297-0972-4bad-a4f4-ea260dd620ca","Type":"ContainerStarted","Data":"17858977f5bc16fd02589f0d4585dc78788d02c65bf5dbc792ebfbf0cd8a17c6"}
Sep 11 13:58:40.600350 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:40.600334    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:58:40.631562 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:40.631521    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vz6bg" podStartSLOduration=64.97416207 podStartE2EDuration="1m9.631508218s" totalImagesPullingTime="4.657346148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:40.629714637 +0000 UTC m=+70.967290197" watchObservedRunningTime="2026-09-11 13:58:40.631508218 +0000 UTC m=+70.969083818"
Sep 11 13:58:49.319156 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.319101    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r48dd" podStartSLOduration=74.779431004 podStartE2EDuration="1m19.319082529s" totalImagesPullingTime="4.539651525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:40.65784689 +0000 UTC m=+70.995422450" watchObservedRunningTime="2026-09-11 13:58:49.319082529 +0000 UTC m=+79.656658086"
Sep 11 13:58:49.319962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.319944    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"]
Sep 11 13:58:49.323759 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.323743    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.327343 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.327322    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 13:58:49.327750 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.327732    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 13:58:49.329652 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.329609    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 13:58:49.329732 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.329688    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-7zzk6\""
Sep 11 13:58:49.329785 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.329693    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 13:58:49.337682 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.337663    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"]
Sep 11 13:58:49.355124 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.355104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.355231 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.355173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b357ccb1-6082-4553-8552-322e646338da-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.355295 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.355233    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4smmr\" (UniqueName: \"kubernetes.io/projected/b357ccb1-6082-4553-8552-322e646338da-kube-api-access-4smmr\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.456052 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.456031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b357ccb1-6082-4553-8552-322e646338da-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.456155 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.456058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4smmr\" (UniqueName: \"kubernetes.io/projected/b357ccb1-6082-4553-8552-322e646338da-kube-api-access-4smmr\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.456155 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.456090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.456228 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:49.456199    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:49.456271 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:49.456261    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls podName:b357ccb1-6082-4553-8552-322e646338da nodeName:}" failed. No retries permitted until 2026-09-11 13:58:49.956244752 +0000 UTC m=+80.293820290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vjvrz" (UID: "b357ccb1-6082-4553-8552-322e646338da") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:49.456756 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.456739    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b357ccb1-6082-4553-8552-322e646338da-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.474791 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.474766    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4smmr\" (UniqueName: \"kubernetes.io/projected/b357ccb1-6082-4553-8552-322e646338da-kube-api-access-4smmr\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.960661 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:49.960609    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:49.960859 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:49.960744    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:49.960859 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:49.960807    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls podName:b357ccb1-6082-4553-8552-322e646338da nodeName:}" failed. No retries permitted until 2026-09-11 13:58:50.960793588 +0000 UTC m=+81.298369141 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vjvrz" (UID: "b357ccb1-6082-4553-8552-322e646338da") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:50.967695 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:50.967660    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:50.968083 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:50.967769    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:50.968083 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:50.967836    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls podName:b357ccb1-6082-4553-8552-322e646338da nodeName:}" failed. No retries permitted until 2026-09-11 13:58:52.967819237 +0000 UTC m=+83.305394795 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vjvrz" (UID: "b357ccb1-6082-4553-8552-322e646338da") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:52.981552 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:52.981426    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:52.981945 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:52.981564    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:52.981945 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:52.981644    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls podName:b357ccb1-6082-4553-8552-322e646338da nodeName:}" failed. No retries permitted until 2026-09-11 13:58:56.981613612 +0000 UTC m=+87.319189150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vjvrz" (UID: "b357ccb1-6082-4553-8552-322e646338da") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:53.274259 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.274181    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"]
Sep 11 13:58:53.277270 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.277254    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.281187 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.281158    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 13:58:53.281311 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.281231    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 13:58:53.281476 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.281459    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:58:53.281537 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.281521    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 13:58:53.282055 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.282040    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jsz99\""
Sep 11 13:58:53.288881 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.288859    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"]
Sep 11 13:58:53.384362 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.384335    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a8c46b7-5901-4f1e-b98b-9197917b7656-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-p58zp\" (UID: \"4a8c46b7-5901-4f1e-b98b-9197917b7656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.384510 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.384452    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a8c46b7-5901-4f1e-b98b-9197917b7656-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-p58zp\" (UID: \"4a8c46b7-5901-4f1e-b98b-9197917b7656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.384510 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.384482    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfk8b\" (UniqueName: \"kubernetes.io/projected/4a8c46b7-5901-4f1e-b98b-9197917b7656-kube-api-access-qfk8b\") pod \"kube-storage-version-migrator-operator-6c8f84f897-p58zp\" (UID: \"4a8c46b7-5901-4f1e-b98b-9197917b7656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.484770 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.484731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a8c46b7-5901-4f1e-b98b-9197917b7656-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-p58zp\" (UID: \"4a8c46b7-5901-4f1e-b98b-9197917b7656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.484866 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.484824    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a8c46b7-5901-4f1e-b98b-9197917b7656-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-p58zp\" (UID: \"4a8c46b7-5901-4f1e-b98b-9197917b7656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.484866 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.484844    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfk8b\" (UniqueName: \"kubernetes.io/projected/4a8c46b7-5901-4f1e-b98b-9197917b7656-kube-api-access-qfk8b\") pod \"kube-storage-version-migrator-operator-6c8f84f897-p58zp\" (UID: \"4a8c46b7-5901-4f1e-b98b-9197917b7656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.485368 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.485349    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4a8c46b7-5901-4f1e-b98b-9197917b7656-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-p58zp\" (UID: \"4a8c46b7-5901-4f1e-b98b-9197917b7656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.487057 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.487040    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4a8c46b7-5901-4f1e-b98b-9197917b7656-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-p58zp\" (UID: \"4a8c46b7-5901-4f1e-b98b-9197917b7656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.493264 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.493243    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfk8b\" (UniqueName: \"kubernetes.io/projected/4a8c46b7-5901-4f1e-b98b-9197917b7656-kube-api-access-qfk8b\") pod \"kube-storage-version-migrator-operator-6c8f84f897-p58zp\" (UID: \"4a8c46b7-5901-4f1e-b98b-9197917b7656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.585868 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.585840    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"
Sep 11 13:58:53.856659 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:53.856567    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp"]
Sep 11 13:58:53.859390 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:58:53.859360    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8c46b7_5901_4f1e_b98b_9197917b7656.slice/crio-e629f8b81aa38b1bb0fbb6ac01830173adff4e43331cfd6bc584270e79b052b9 WatchSource:0}: Error finding container e629f8b81aa38b1bb0fbb6ac01830173adff4e43331cfd6bc584270e79b052b9: Status 404 returned error can't find the container with id e629f8b81aa38b1bb0fbb6ac01830173adff4e43331cfd6bc584270e79b052b9
Sep 11 13:58:54.312174 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.312141    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-45kfw"]
Sep 11 13:58:54.315179 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.315153    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.317712 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.317676    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 13:58:54.317886 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.317830    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 13:58:54.317950 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.317905    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 13:58:54.318757 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.318734    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-85l57\""
Sep 11 13:58:54.319747 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.319725    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 13:58:54.332720 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.332698    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-45kfw"]
Sep 11 13:58:54.392183 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.392156    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.392325 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.392186    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.392325 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.392214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwb4l\" (UniqueName: \"kubernetes.io/projected/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-api-access-kwb4l\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.392325 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.392281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-data-volume\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.392450 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.392436    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-crio-socket\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.492815 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.492783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-crio-socket\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.492962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.492825    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.492962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.492857    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.492962 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:54.492931    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:58:54.492962 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:54.492953    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:58:54.493187 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:54.493000    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:54.992984537 +0000 UTC m=+85.330560094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : configmap "kube-rbac-proxy" not found
Sep 11 13:58:54.493187 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.493006    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-crio-socket\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.493187 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:54.493014    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:54.993008405 +0000 UTC m=+85.330583943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:58:54.493187 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.493057    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwb4l\" (UniqueName: \"kubernetes.io/projected/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-api-access-kwb4l\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.493187 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.493086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-data-volume\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.493453 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.493418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-data-volume\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.505815 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.505790    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwb4l\" (UniqueName: \"kubernetes.io/projected/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-api-access-kwb4l\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.637268 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.637188    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp" event={"ID":"4a8c46b7-5901-4f1e-b98b-9197917b7656","Type":"ContainerStarted","Data":"e629f8b81aa38b1bb0fbb6ac01830173adff4e43331cfd6bc584270e79b052b9"}
Sep 11 13:58:54.998053 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.997907    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.998053 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:54.997953    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:54.998274 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:54.998053    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:58:54.998274 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:54.998076    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:58:54.998274 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:54.998134    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:55.998115258 +0000 UTC m=+86.335690801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : configmap "kube-rbac-proxy" not found
Sep 11 13:58:54.998274 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:54.998155    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:55.998146157 +0000 UTC m=+86.335721697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:58:56.006311 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.006216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:56.006311 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.006269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:56.006740 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:56.006337    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:58:56.006740 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:56.006385    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:58:56.006740 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:56.006415    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:58.00640013 +0000 UTC m=+88.343975668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : configmap "kube-rbac-proxy" not found
Sep 11 13:58:56.006740 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:56.006436    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:58:58.006421319 +0000 UTC m=+88.343996863 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:58:56.030057 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.030034    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2zmjn_bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa/dns-node-resolver/0.log"
Sep 11 13:58:56.261264 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.261190    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"]
Sep 11 13:58:56.264210 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.264195    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.268049 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.268029    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 13:58:56.268432 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.268411    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 13:58:56.268547 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.268531    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 13:58:56.268956 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.268940    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vhd5g\""
Sep 11 13:58:56.269156 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.269137    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:58:56.275877 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.275856    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"]
Sep 11 13:58:56.309404 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.309378    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bcaf3c33-d28c-435e-9901-c977c195aa0e-serving-cert\") pod \"service-ca-operator-cf7c776b7-xphn5\" (UID: \"bcaf3c33-d28c-435e-9901-c977c195aa0e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.309495 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.309480    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bcaf3c33-d28c-435e-9901-c977c195aa0e-config\") pod \"service-ca-operator-cf7c776b7-xphn5\" (UID: \"bcaf3c33-d28c-435e-9901-c977c195aa0e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.309540 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.309508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gbq8\" (UniqueName: \"kubernetes.io/projected/bcaf3c33-d28c-435e-9901-c977c195aa0e-kube-api-access-7gbq8\") pod \"service-ca-operator-cf7c776b7-xphn5\" (UID: \"bcaf3c33-d28c-435e-9901-c977c195aa0e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.410880 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.410846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bcaf3c33-d28c-435e-9901-c977c195aa0e-config\") pod \"service-ca-operator-cf7c776b7-xphn5\" (UID: \"bcaf3c33-d28c-435e-9901-c977c195aa0e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.410880 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.410883    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gbq8\" (UniqueName: \"kubernetes.io/projected/bcaf3c33-d28c-435e-9901-c977c195aa0e-kube-api-access-7gbq8\") pod \"service-ca-operator-cf7c776b7-xphn5\" (UID: \"bcaf3c33-d28c-435e-9901-c977c195aa0e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.411022 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.410940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bcaf3c33-d28c-435e-9901-c977c195aa0e-serving-cert\") pod \"service-ca-operator-cf7c776b7-xphn5\" (UID: \"bcaf3c33-d28c-435e-9901-c977c195aa0e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.411573 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.411551    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bcaf3c33-d28c-435e-9901-c977c195aa0e-config\") pod \"service-ca-operator-cf7c776b7-xphn5\" (UID: \"bcaf3c33-d28c-435e-9901-c977c195aa0e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.413115 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.413095    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bcaf3c33-d28c-435e-9901-c977c195aa0e-serving-cert\") pod \"service-ca-operator-cf7c776b7-xphn5\" (UID: \"bcaf3c33-d28c-435e-9901-c977c195aa0e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.420065 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.420044    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gbq8\" (UniqueName: \"kubernetes.io/projected/bcaf3c33-d28c-435e-9901-c977c195aa0e-kube-api-access-7gbq8\") pod \"service-ca-operator-cf7c776b7-xphn5\" (UID: \"bcaf3c33-d28c-435e-9901-c977c195aa0e\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.573700 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.573671    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"
Sep 11 13:58:56.644713 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.644687    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp" event={"ID":"4a8c46b7-5901-4f1e-b98b-9197917b7656","Type":"ContainerStarted","Data":"96422b1ffc5be3374486254870e61ea79aa25c665411c64702786b77fd4b7484"}
Sep 11 13:58:56.703787 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.703728    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp" podStartSLOduration=1.9424823610000002 podStartE2EDuration="3.703709315s" totalImagesPullingTime="1.761226954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:56.703146406 +0000 UTC m=+87.040721965" watchObservedRunningTime="2026-09-11 13:58:56.703709315 +0000 UTC m=+87.041284877"
Sep 11 13:58:56.839095 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:56.839021    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5"]
Sep 11 13:58:56.841680 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:58:56.841653    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcaf3c33_d28c_435e_9901_c977c195aa0e.slice/crio-ebae04fe46d7fac4f27223f44df9a37989114b3e18740efff34b423e581e5abf WatchSource:0}: Error finding container ebae04fe46d7fac4f27223f44df9a37989114b3e18740efff34b423e581e5abf: Status 404 returned error can't find the container with id ebae04fe46d7fac4f27223f44df9a37989114b3e18740efff34b423e581e5abf
Sep 11 13:58:57.016993 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:57.016963    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:58:57.017340 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:57.017065    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:57.017340 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:57.017114    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls podName:b357ccb1-6082-4553-8552-322e646338da nodeName:}" failed. No retries permitted until 2026-09-11 13:59:05.017100981 +0000 UTC m=+95.354676519 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vjvrz" (UID: "b357ccb1-6082-4553-8552-322e646338da") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:58:57.225643 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:57.225564    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hjsjr_00dc4e9d-f4e7-41a6-9d88-3411a8df5400/node-ca/0.log"
Sep 11 13:58:57.648516 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:57.648470    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5" event={"ID":"bcaf3c33-d28c-435e-9901-c977c195aa0e","Type":"ContainerStarted","Data":"ebae04fe46d7fac4f27223f44df9a37989114b3e18740efff34b423e581e5abf"}
Sep 11 13:58:58.027808 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:58.027720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:58.027808 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:58.027762    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:58:58.028236 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:58.027843    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:58:58.028236 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:58.027891    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:58:58.028236 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:58.027972    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:59:02.027895155 +0000 UTC m=+92.365470717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : configmap "kube-rbac-proxy" not found
Sep 11 13:58:58.028236 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:58.027992    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:59:02.027982674 +0000 UTC m=+92.365558212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:58:58.030477 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:58.030422    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k6t77_2af3aa4c-d665-483e-874c-0d085d6d1da9/serve-healthcheck-canary/0.log"
Sep 11 13:58:59.354209 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.354177    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"]
Sep 11 13:58:59.357079 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.357062    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:58:59.359638 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.359609    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 13:58:59.359747 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.359708    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-m6tvp\""
Sep 11 13:58:59.360048 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.360032    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 13:58:59.368684 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.368660    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"]
Sep 11 13:58:59.439291 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.439265    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:58:59.439388 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.439336    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:58:59.540012 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.539983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:58:59.540182 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.540166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:58:59.540330 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:59.540310    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:58:59.540400 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:58:59.540388    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert podName:c4e76c34-0414-47ad-971f-2e3aaad3ec4a nodeName:}" failed. No retries permitted until 2026-09-11 13:59:00.040365691 +0000 UTC m=+90.377941242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-r4547" (UID: "c4e76c34-0414-47ad-971f-2e3aaad3ec4a") : secret "networking-console-plugin-cert" not found
Sep 11 13:58:59.540736 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.540721    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:58:59.655977 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.655903    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5" event={"ID":"bcaf3c33-d28c-435e-9901-c977c195aa0e","Type":"ContainerStarted","Data":"ac5cbdc1ab9ff5d1205661607d32fb9a12aff20e9ee342e64e1c20393c966dc9"}
Sep 11 13:58:59.675689 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:58:59.675617    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5" podStartSLOduration=1.551590223 podStartE2EDuration="3.675603357s" totalImagesPullingTime="2.124013134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:58:59.674964759 +0000 UTC m=+90.012540367" watchObservedRunningTime="2026-09-11 13:58:59.675603357 +0000 UTC m=+90.013178999"
Sep 11 13:59:00.045503 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:00.045424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:59:00.045640 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:00.045588    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:59:00.045689 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:00.045669    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert podName:c4e76c34-0414-47ad-971f-2e3aaad3ec4a nodeName:}" failed. No retries permitted until 2026-09-11 13:59:01.045652615 +0000 UTC m=+91.383228153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-r4547" (UID: "c4e76c34-0414-47ad-971f-2e3aaad3ec4a") : secret "networking-console-plugin-cert" not found
Sep 11 13:59:01.054862 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:01.054825    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:59:01.055237 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:01.054973    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:59:01.055237 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:01.055054    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert podName:c4e76c34-0414-47ad-971f-2e3aaad3ec4a nodeName:}" failed. No retries permitted until 2026-09-11 13:59:03.055032573 +0000 UTC m=+93.392608128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-r4547" (UID: "c4e76c34-0414-47ad-971f-2e3aaad3ec4a") : secret "networking-console-plugin-cert" not found
Sep 11 13:59:02.063991 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.063958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:59:02.064362 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.064019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:59:02.064362 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:02.064076    2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:59:02.064362 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:02.064109    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:59:02.064362 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:02.064140    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:59:10.064125135 +0000 UTC m=+100.401700677 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : configmap "kube-rbac-proxy" not found
Sep 11 13:59:02.064362 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:02.064155    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:59:10.06414886 +0000 UTC m=+100.401724401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:59:02.611919 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.611888    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-5rj8v"]
Sep 11 13:59:02.614920 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.614905    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.617295 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.617273    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 13:59:02.617406 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.617273    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 13:59:02.618019 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.618003    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 13:59:02.618072 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.618027    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 13:59:02.619615 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.619601    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wk82s\""
Sep 11 13:59:02.626591 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.626571    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-5rj8v"]
Sep 11 13:59:02.667692 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.667667    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kdrp\" (UniqueName: \"kubernetes.io/projected/7e8b3b68-5b4c-4188-a2a5-16da11264afe-kube-api-access-2kdrp\") pod \"service-ca-568869fdc5-5rj8v\" (UID: \"7e8b3b68-5b4c-4188-a2a5-16da11264afe\") " pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.667778 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.667699    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7e8b3b68-5b4c-4188-a2a5-16da11264afe-signing-cabundle\") pod \"service-ca-568869fdc5-5rj8v\" (UID: \"7e8b3b68-5b4c-4188-a2a5-16da11264afe\") " pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.667778 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.667739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7e8b3b68-5b4c-4188-a2a5-16da11264afe-signing-key\") pod \"service-ca-568869fdc5-5rj8v\" (UID: \"7e8b3b68-5b4c-4188-a2a5-16da11264afe\") " pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.768180 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.768157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7e8b3b68-5b4c-4188-a2a5-16da11264afe-signing-key\") pod \"service-ca-568869fdc5-5rj8v\" (UID: \"7e8b3b68-5b4c-4188-a2a5-16da11264afe\") " pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.768277 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.768251    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kdrp\" (UniqueName: \"kubernetes.io/projected/7e8b3b68-5b4c-4188-a2a5-16da11264afe-kube-api-access-2kdrp\") pod \"service-ca-568869fdc5-5rj8v\" (UID: \"7e8b3b68-5b4c-4188-a2a5-16da11264afe\") " pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.768325 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.768278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7e8b3b68-5b4c-4188-a2a5-16da11264afe-signing-cabundle\") pod \"service-ca-568869fdc5-5rj8v\" (UID: \"7e8b3b68-5b4c-4188-a2a5-16da11264afe\") " pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.768883 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.768865    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7e8b3b68-5b4c-4188-a2a5-16da11264afe-signing-cabundle\") pod \"service-ca-568869fdc5-5rj8v\" (UID: \"7e8b3b68-5b4c-4188-a2a5-16da11264afe\") " pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.770481 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.770465    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7e8b3b68-5b4c-4188-a2a5-16da11264afe-signing-key\") pod \"service-ca-568869fdc5-5rj8v\" (UID: \"7e8b3b68-5b4c-4188-a2a5-16da11264afe\") " pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.779909 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.779883    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kdrp\" (UniqueName: \"kubernetes.io/projected/7e8b3b68-5b4c-4188-a2a5-16da11264afe-kube-api-access-2kdrp\") pod \"service-ca-568869fdc5-5rj8v\" (UID: \"7e8b3b68-5b4c-4188-a2a5-16da11264afe\") " pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:02.923142 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:02.923060    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-5rj8v"
Sep 11 13:59:03.070515 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:03.070482    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:59:03.070976 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:03.070684    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:59:03.070976 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:03.070766    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert podName:c4e76c34-0414-47ad-971f-2e3aaad3ec4a nodeName:}" failed. No retries permitted until 2026-09-11 13:59:07.070743046 +0000 UTC m=+97.408318598 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-r4547" (UID: "c4e76c34-0414-47ad-971f-2e3aaad3ec4a") : secret "networking-console-plugin-cert" not found
Sep 11 13:59:03.191274 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:03.191247    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-5rj8v"]
Sep 11 13:59:03.193225 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:03.193198    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8b3b68_5b4c_4188_a2a5_16da11264afe.slice/crio-0c7844a5a93367ab7bdd5da512b96b71a4b170c95ff1a6087a33d71b1413098c WatchSource:0}: Error finding container 0c7844a5a93367ab7bdd5da512b96b71a4b170c95ff1a6087a33d71b1413098c: Status 404 returned error can't find the container with id 0c7844a5a93367ab7bdd5da512b96b71a4b170c95ff1a6087a33d71b1413098c
Sep 11 13:59:03.667748 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:03.667713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-5rj8v" event={"ID":"7e8b3b68-5b4c-4188-a2a5-16da11264afe","Type":"ContainerStarted","Data":"188f55b24ee09d7e58dd2b6c95931c2b9a83bc74c31964806256384350335bfd"}
Sep 11 13:59:03.667903 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:03.667790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-5rj8v" event={"ID":"7e8b3b68-5b4c-4188-a2a5-16da11264afe","Type":"ContainerStarted","Data":"0c7844a5a93367ab7bdd5da512b96b71a4b170c95ff1a6087a33d71b1413098c"}
Sep 11 13:59:03.694672 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:03.694602    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-5rj8v" podStartSLOduration=1.694581992 podStartE2EDuration="1.694581992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:03.692311352 +0000 UTC m=+94.029886911" watchObservedRunningTime="2026-09-11 13:59:03.694581992 +0000 UTC m=+94.032157553"
Sep 11 13:59:05.087097 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:05.087061    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:59:05.087517 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:05.087241    2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 13:59:05.087517 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:05.087310    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls podName:b357ccb1-6082-4553-8552-322e646338da nodeName:}" failed. No retries permitted until 2026-09-11 13:59:21.087293996 +0000 UTC m=+111.424869547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-vjvrz" (UID: "b357ccb1-6082-4553-8552-322e646338da") : secret "cluster-monitoring-operator-tls" not found
Sep 11 13:59:07.105197 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:07.105157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:59:07.105690 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:07.105310    2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 13:59:07.105690 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:07.105384    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert podName:c4e76c34-0414-47ad-971f-2e3aaad3ec4a nodeName:}" failed. No retries permitted until 2026-09-11 13:59:15.105368746 +0000 UTC m=+105.442944297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-r4547" (UID: "c4e76c34-0414-47ad-971f-2e3aaad3ec4a") : secret "networking-console-plugin-cert" not found
Sep 11 13:59:08.921445 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:08.921409    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:59:08.924130 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:08.924104    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"image-registry-d89d5975b-jzjxc\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") " pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:59:08.994951 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:08.994922    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kzr4h\""
Sep 11 13:59:09.003057 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.003031    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:59:09.123164 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.123132    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:59:09.125456 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.125425    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d6a7405c-5ea0-49dc-a61b-c3397ec12438-metrics-tls\") pod \"dns-default-p7bff\" (UID: \"d6a7405c-5ea0-49dc-a61b-c3397ec12438\") " pod="openshift-dns/dns-default-p7bff"
Sep 11 13:59:09.315434 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.315404    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d89d5975b-jzjxc"]
Sep 11 13:59:09.318650 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:09.318614    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63052ae9_d645_4427_9397_527caeb4823f.slice/crio-9aca774ca88690fb77375fd4b191473c1099ea24b522bd96cdd7b54dbee791c5 WatchSource:0}: Error finding container 9aca774ca88690fb77375fd4b191473c1099ea24b522bd96cdd7b54dbee791c5: Status 404 returned error can't find the container with id 9aca774ca88690fb77375fd4b191473c1099ea24b522bd96cdd7b54dbee791c5
Sep 11 13:59:09.407085 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.407063    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tbm52\""
Sep 11 13:59:09.414833 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.414812    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p7bff"
Sep 11 13:59:09.689193 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.689113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc" event={"ID":"63052ae9-d645-4427-9397-527caeb4823f","Type":"ContainerStarted","Data":"c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f"}
Sep 11 13:59:09.689193 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.689165    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc" event={"ID":"63052ae9-d645-4427-9397-527caeb4823f","Type":"ContainerStarted","Data":"9aca774ca88690fb77375fd4b191473c1099ea24b522bd96cdd7b54dbee791c5"}
Sep 11 13:59:09.689385 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.689244    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:59:09.739807 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.739642    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p7bff"]
Sep 11 13:59:09.742375 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:09.742350    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a7405c_5ea0_49dc_a61b_c3397ec12438.slice/crio-aab0558f13c0b54035729cdbe0c236c034e0403445843d4e9ba67049daaa089e WatchSource:0}: Error finding container aab0558f13c0b54035729cdbe0c236c034e0403445843d4e9ba67049daaa089e: Status 404 returned error can't find the container with id aab0558f13c0b54035729cdbe0c236c034e0403445843d4e9ba67049daaa089e
Sep 11 13:59:09.750120 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:09.750082    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc" podStartSLOduration=98.750066401 podStartE2EDuration="1m38.750066401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:09.749953456 +0000 UTC m=+100.087529053" watchObservedRunningTime="2026-09-11 13:59:09.750066401 +0000 UTC m=+100.087641961"
Sep 11 13:59:10.133818 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:10.133785    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:59:10.134181 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:10.133849    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:59:10.134181 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:10.133956    2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:59:10.134181 ip-10-0-129-195 kubenswrapper[2564]: E0911 13:59:10.134017    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls podName:1f4e7bd1-2741-48b2-a850-7ae70e22ad87 nodeName:}" failed. No retries permitted until 2026-09-11 13:59:26.133995991 +0000 UTC m=+116.471571549 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls") pod "insights-runtime-extractor-45kfw" (UID: "1f4e7bd1-2741-48b2-a850-7ae70e22ad87") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:59:10.134323 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:10.134299    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:59:10.694255 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:10.694202    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p7bff" event={"ID":"d6a7405c-5ea0-49dc-a61b-c3397ec12438","Type":"ContainerStarted","Data":"aab0558f13c0b54035729cdbe0c236c034e0403445843d4e9ba67049daaa089e"}
Sep 11 13:59:11.605443 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:11.605407    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r48dd"
Sep 11 13:59:12.702017 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:12.701973    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p7bff" event={"ID":"d6a7405c-5ea0-49dc-a61b-c3397ec12438","Type":"ContainerStarted","Data":"61ed25ee6515d3eaf9c534512ed8821297fd91492c9b719d776f92b07d375a48"}
Sep 11 13:59:12.702407 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:12.702048    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p7bff" event={"ID":"d6a7405c-5ea0-49dc-a61b-c3397ec12438","Type":"ContainerStarted","Data":"0bea7b3f0da9fddbc41776d1e40b9d9ccc41605489b109e5c94b4117f23f97ab"}
Sep 11 13:59:12.702407 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:12.702079    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p7bff"
Sep 11 13:59:12.726861 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:12.726812    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-p7bff" podStartSLOduration=65.599073459 podStartE2EDuration="1m7.72679554s" totalImagesPullingTime="2.127722081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:12.726288686 +0000 UTC m=+103.063864247" watchObservedRunningTime="2026-09-11 13:59:12.72679554 +0000 UTC m=+103.064371134"
Sep 11 13:59:15.176317 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:15.176266    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:59:15.178637 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:15.178599    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c4e76c34-0414-47ad-971f-2e3aaad3ec4a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-r4547\" (UID: \"c4e76c34-0414-47ad-971f-2e3aaad3ec4a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:59:15.265913 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:15.265872    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"
Sep 11 13:59:15.539861 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:15.539780    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547"]
Sep 11 13:59:15.542887 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:15.542864    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4e76c34_0414_47ad_971f_2e3aaad3ec4a.slice/crio-b3f135b281e064e6a40825ef19308cd9ecfdbe82e7c4558712088f5c3d64446a WatchSource:0}: Error finding container b3f135b281e064e6a40825ef19308cd9ecfdbe82e7c4558712088f5c3d64446a: Status 404 returned error can't find the container with id b3f135b281e064e6a40825ef19308cd9ecfdbe82e7c4558712088f5c3d64446a
Sep 11 13:59:15.713525 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:15.713493    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547" event={"ID":"c4e76c34-0414-47ad-971f-2e3aaad3ec4a","Type":"ContainerStarted","Data":"b3f135b281e064e6a40825ef19308cd9ecfdbe82e7c4558712088f5c3d64446a"}
Sep 11 13:59:16.717529 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:16.717494    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547" event={"ID":"c4e76c34-0414-47ad-971f-2e3aaad3ec4a","Type":"ContainerStarted","Data":"e895646cefc9d10af7b42fa5aa04fc478a8ba404130a9bf6af90a055c59cb703"}
Sep 11 13:59:16.734741 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:16.734697    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-r4547" podStartSLOduration=16.64629556 podStartE2EDuration="17.734680697s" totalImagesPullingTime="1.088385137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:16.733837045 +0000 UTC m=+107.071412607" watchObservedRunningTime="2026-09-11 13:59:16.734680697 +0000 UTC m=+107.072256262"
Sep 11 13:59:21.121794 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:21.121733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:59:21.124184 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:21.124148    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b357ccb1-6082-4553-8552-322e646338da-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-vjvrz\" (UID: \"b357ccb1-6082-4553-8552-322e646338da\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:59:21.133012 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:21.132991    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"
Sep 11 13:59:21.401323 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:21.401215    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz"]
Sep 11 13:59:21.403906 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:21.403874    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb357ccb1_6082_4553_8552_322e646338da.slice/crio-9e4b71eda2e318f2e8447706caa88aef84164586b3152ba9352c81636092b06d WatchSource:0}: Error finding container 9e4b71eda2e318f2e8447706caa88aef84164586b3152ba9352c81636092b06d: Status 404 returned error can't find the container with id 9e4b71eda2e318f2e8447706caa88aef84164586b3152ba9352c81636092b06d
Sep 11 13:59:21.736510 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:21.736423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz" event={"ID":"b357ccb1-6082-4553-8552-322e646338da","Type":"ContainerStarted","Data":"9e4b71eda2e318f2e8447706caa88aef84164586b3152ba9352c81636092b06d"}
Sep 11 13:59:22.708639 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:22.708593    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p7bff"
Sep 11 13:59:23.744994 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:23.744955    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz" event={"ID":"b357ccb1-6082-4553-8552-322e646338da","Type":"ContainerStarted","Data":"f6f1f2ff05a7d6f7fa7f79c860584bc3419fb93401db88d88b7d6e2508f23b95"}
Sep 11 13:59:26.159213 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:26.159170    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:59:26.161529 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:26.161509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1f4e7bd1-2741-48b2-a850-7ae70e22ad87-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-45kfw\" (UID: \"1f4e7bd1-2741-48b2-a850-7ae70e22ad87\") " pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:59:26.425696 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:26.425586    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-45kfw"
Sep 11 13:59:26.696315 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:26.696204    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-vjvrz" podStartSLOduration=36.115644673 podStartE2EDuration="37.696185486s" totalImagesPullingTime="1.580540813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:23.921593983 +0000 UTC m=+114.259169543" watchObservedRunningTime="2026-09-11 13:59:26.696185486 +0000 UTC m=+117.033761041"
Sep 11 13:59:26.697067 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:26.697048    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-45kfw"]
Sep 11 13:59:26.700202 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:26.700179    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4e7bd1_2741_48b2_a850_7ae70e22ad87.slice/crio-beaef16122c2af036593da3af4728f6d1a9daab9f398c985ac20988eee410783 WatchSource:0}: Error finding container beaef16122c2af036593da3af4728f6d1a9daab9f398c985ac20988eee410783: Status 404 returned error can't find the container with id beaef16122c2af036593da3af4728f6d1a9daab9f398c985ac20988eee410783
Sep 11 13:59:26.755519 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:26.755494    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-45kfw" event={"ID":"1f4e7bd1-2741-48b2-a850-7ae70e22ad87","Type":"ContainerStarted","Data":"beaef16122c2af036593da3af4728f6d1a9daab9f398c985ac20988eee410783"}
Sep 11 13:59:27.760718 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:27.760684    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-45kfw" event={"ID":"1f4e7bd1-2741-48b2-a850-7ae70e22ad87","Type":"ContainerStarted","Data":"cca99923f76c5819fa283669ba3526596a01d43941dd39de86a88ac3c255f2c9"}
Sep 11 13:59:27.761070 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:27.760744    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-45kfw" event={"ID":"1f4e7bd1-2741-48b2-a850-7ae70e22ad87","Type":"ContainerStarted","Data":"a3aee63a040f32f63c179e3fde89b00865d44624fba5610a79ef0d92e463ee01"}
Sep 11 13:59:29.007759 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:29.007718    2564 patch_prober.go:28] interesting pod/image-registry-d89d5975b-jzjxc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 13:59:29.008097 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:29.007767    2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc" podUID="63052ae9-d645-4427-9397-527caeb4823f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:59:29.770837 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:29.770797    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-45kfw" event={"ID":"1f4e7bd1-2741-48b2-a850-7ae70e22ad87","Type":"ContainerStarted","Data":"14a9bf90a255084972aa6d1d77ac80ff2424954812d68c8df7637344fd22b6f8"}
Sep 11 13:59:29.794165 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:29.794116    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-45kfw" podStartSLOduration=33.754163058 podStartE2EDuration="35.794101454s" totalImagesPullingTime="2.039938396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:58:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:29.792948638 +0000 UTC m=+120.130524224" watchObservedRunningTime="2026-09-11 13:59:29.794101454 +0000 UTC m=+120.131677013"
Sep 11 13:59:30.698322 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:30.698295    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 13:59:37.259227 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.259194    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l8fs8"]
Sep 11 13:59:37.264767 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.264746    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"]
Sep 11 13:59:37.264916 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.264894    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.267394 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.267366    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 13:59:37.267510 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.267418    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 13:59:37.268086 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.268065    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 13:59:37.268172 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.268156    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 13:59:37.268431 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.268417    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wckx2\""
Sep 11 13:59:37.268776 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.268756    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.272311 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.272291    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 11 13:59:37.272421 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.272325    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 11 13:59:37.272684 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.272666    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-lrh6f\""
Sep 11 13:59:37.272802 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.272782    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 11 13:59:37.277698 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.277677    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"]
Sep 11 13:59:37.442599 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442565    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfslk\" (UniqueName: \"kubernetes.io/projected/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-kube-api-access-bfslk\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.442799 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442642    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-metrics-client-ca\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.442799 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442737    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-sys\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.442799 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442788    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-accelerators-collector-config\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.442962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442831    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.442962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.442962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442902    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.442962 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-wtmp\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.443166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442963    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cr2z\" (UniqueName: \"kubernetes.io/projected/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-api-access-9cr2z\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.443166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.442990    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.443166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.443028    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-textfile\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.443166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.443052    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-tls\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.443166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.443109    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-root\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.443166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.443134    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.443509 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.443166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.543863 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.543785    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-textfile\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.543863 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.543828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-tls\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.543863 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.543847    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-root\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.543863 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.543864    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.544165 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.543883    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.544165 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.543911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfslk\" (UniqueName: \"kubernetes.io/projected/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-kube-api-access-bfslk\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544165 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.543938    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-metrics-client-ca\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544165 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.543985    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-root\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544362 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544183    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-textfile\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544362 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-sys\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544362 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544267    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-accelerators-collector-config\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544362 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544281    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-sys\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544362 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.544362 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544358    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.544700 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544395    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.544700 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544435    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-wtmp\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544700 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544459    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cr2z\" (UniqueName: \"kubernetes.io/projected/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-api-access-9cr2z\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.544700 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544489    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544700 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544534    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-metrics-client-ca\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.544951 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544855    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.544951 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.544884    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-accelerators-collector-config\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.545056 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.545026    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-wtmp\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.545254 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.545155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.545413 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.545392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.546806 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.546781    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.546921 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.546808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.546921 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.546913    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-node-exporter-tls\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.547214 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.547199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.552259 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.552232    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfslk\" (UniqueName: \"kubernetes.io/projected/ce92e2e1-1ebc-4f45-afda-0d7f2086b883-kube-api-access-bfslk\") pod \"node-exporter-l8fs8\" (UID: \"ce92e2e1-1ebc-4f45-afda-0d7f2086b883\") " pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.553315 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.553294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cr2z\" (UniqueName: \"kubernetes.io/projected/1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c-kube-api-access-9cr2z\") pod \"kube-state-metrics-59fbdd77d-hbcz5\" (UID: \"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.577001 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.576977    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l8fs8"
Sep 11 13:59:37.583154 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.583137    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"
Sep 11 13:59:37.594839 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:37.594818    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce92e2e1_1ebc_4f45_afda_0d7f2086b883.slice/crio-60e99f2893d7edf529b4b54f45172be99182c191b17fc063b9692e6b853dbffa WatchSource:0}: Error finding container 60e99f2893d7edf529b4b54f45172be99182c191b17fc063b9692e6b853dbffa: Status 404 returned error can't find the container with id 60e99f2893d7edf529b4b54f45172be99182c191b17fc063b9692e6b853dbffa
Sep 11 13:59:37.795366 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.795298    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l8fs8" event={"ID":"ce92e2e1-1ebc-4f45-afda-0d7f2086b883","Type":"ContainerStarted","Data":"60e99f2893d7edf529b4b54f45172be99182c191b17fc063b9692e6b853dbffa"}
Sep 11 13:59:37.859696 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:37.859667    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5"]
Sep 11 13:59:37.862302 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:37.862274    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aef610a_7ea0_477f_97a5_7f8ea0c4fc6c.slice/crio-2c6e7d3bcdfd0edfca125622ffff09ed69f2fa00e40072ddcfdd822a12a69eb5 WatchSource:0}: Error finding container 2c6e7d3bcdfd0edfca125622ffff09ed69f2fa00e40072ddcfdd822a12a69eb5: Status 404 returned error can't find the container with id 2c6e7d3bcdfd0edfca125622ffff09ed69f2fa00e40072ddcfdd822a12a69eb5
Sep 11 13:59:38.033398 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.033361    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:59:38.038874 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.038847    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.042508 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.042484    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 13:59:38.043293 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.043275    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 13:59:38.043396 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.043294    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 13:59:38.043396 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.043309    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 13:59:38.043607 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.043580    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 13:59:38.043607 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.043581    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 13:59:38.044065 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.044038    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sgqtp\""
Sep 11 13:59:38.044182 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.044079    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 13:59:38.044243 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.044229    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 13:59:38.052188 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.052173    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 13:59:38.070174 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.070151    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:59:38.150111 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-out\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150283 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150121    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150283 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150283 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150247    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150283 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150275    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150491 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150298    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6c2x\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-kube-api-access-f6c2x\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150491 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150360    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150491 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150414    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150491 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150450    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150707 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150511    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150707 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150594    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150707 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-web-config\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.150707 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.150686    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-volume\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.251812 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.251774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.251986 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.251837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.251986 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.251872    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.251986 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.251913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.251986 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.251948    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-web-config\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.251986 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.251973    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-volume\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.252166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.252002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-out\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.252166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.252024    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.252166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.252058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.252166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.252086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.252166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.252109    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.252166 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.252131    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6c2x\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-kube-api-access-f6c2x\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.252411 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.252174    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.252713 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.252601    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.253934 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.253434    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.253934 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.253682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.255995 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.255967    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.256584 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.256477    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.256584 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.256543    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.256855 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.256830    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.256928 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.256916    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.257105 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.257083    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-out\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.257286 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.257265    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.257458 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.257434    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-web-config\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.258100 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.258078    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-volume\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.262492 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.262473    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6c2x\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-kube-api-access-f6c2x\") pod \"alertmanager-main-0\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.348309 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.348281    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 13:59:38.801307 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.801274    2564 generic.go:356] "Generic (PLEG): container finished" podID="ce92e2e1-1ebc-4f45-afda-0d7f2086b883" containerID="5e8c6930505d4297aecec07a300e4fc893e62b5c6ccfb46339049f9c62ee9850" exitCode=0
Sep 11 13:59:38.801456 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.801374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l8fs8" event={"ID":"ce92e2e1-1ebc-4f45-afda-0d7f2086b883","Type":"ContainerDied","Data":"5e8c6930505d4297aecec07a300e4fc893e62b5c6ccfb46339049f9c62ee9850"}
Sep 11 13:59:38.803221 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.803194    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5" event={"ID":"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c","Type":"ContainerStarted","Data":"2c6e7d3bcdfd0edfca125622ffff09ed69f2fa00e40072ddcfdd822a12a69eb5"}
Sep 11 13:59:38.854063 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:38.853846    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 13:59:39.060376 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.060336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:59:39.062973 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.062941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73e95838-8a95-4047-b739-0f740d140756-metrics-certs\") pod \"network-metrics-daemon-sjsjk\" (UID: \"73e95838-8a95-4047-b739-0f740d140756\") " pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:59:39.138878 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:39.138851    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe9051ee_2d26_44e8_87cf_774b18faa44b.slice/crio-2aabc53b2007b043b71c62db2930dff4de7888a57d4be6cbaaa85843e554c6d6 WatchSource:0}: Error finding container 2aabc53b2007b043b71c62db2930dff4de7888a57d4be6cbaaa85843e554c6d6: Status 404 returned error can't find the container with id 2aabc53b2007b043b71c62db2930dff4de7888a57d4be6cbaaa85843e554c6d6
Sep 11 13:59:39.281584 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.281560    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qh858\""
Sep 11 13:59:39.290418 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.290394    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sjsjk"
Sep 11 13:59:39.582231 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.582180    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sjsjk"]
Sep 11 13:59:39.584515 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:39.584487    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e95838_8a95_4047_b739_0f740d140756.slice/crio-096293f3760a81a46d6b379c2c2edb064cf3015d1b49479ad7de83f3f2d18b88 WatchSource:0}: Error finding container 096293f3760a81a46d6b379c2c2edb064cf3015d1b49479ad7de83f3f2d18b88: Status 404 returned error can't find the container with id 096293f3760a81a46d6b379c2c2edb064cf3015d1b49479ad7de83f3f2d18b88
Sep 11 13:59:39.810186 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.810152    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l8fs8" event={"ID":"ce92e2e1-1ebc-4f45-afda-0d7f2086b883","Type":"ContainerStarted","Data":"cd6fadabf809d8e69569d24d6ab3b6d018e74429cb04aedd1c89b524eeb9a42a"}
Sep 11 13:59:39.810345 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.810211    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l8fs8" event={"ID":"ce92e2e1-1ebc-4f45-afda-0d7f2086b883","Type":"ContainerStarted","Data":"91626ecf6492ca6231c2c08fef022405540724384bad787352ce3a617351f641"}
Sep 11 13:59:39.812700 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.812671    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5" event={"ID":"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c","Type":"ContainerStarted","Data":"04a3ab4d2229cd96acf697d7b26d9248ddab0d95baa07c8b94586142ed76b06a"}
Sep 11 13:59:39.812828 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.812728    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5" event={"ID":"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c","Type":"ContainerStarted","Data":"df0a4e1a7db8f527c9338076672aca56707acf138cee352f7e2cc092e92e1089"}
Sep 11 13:59:39.812828 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.812753    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5" event={"ID":"1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c","Type":"ContainerStarted","Data":"e8ce57a66f78cbce0855543534e9ef859e48cc9adb6ce62426be09eed2c9bf8e"}
Sep 11 13:59:39.814013 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.813985    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sjsjk" event={"ID":"73e95838-8a95-4047-b739-0f740d140756","Type":"ContainerStarted","Data":"096293f3760a81a46d6b379c2c2edb064cf3015d1b49479ad7de83f3f2d18b88"}
Sep 11 13:59:39.815314 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.815292    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerStarted","Data":"2aabc53b2007b043b71c62db2930dff4de7888a57d4be6cbaaa85843e554c6d6"}
Sep 11 13:59:39.829832 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.829647    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l8fs8" podStartSLOduration=1.745331454 podStartE2EDuration="2.829610718s" totalImagesPullingTime="1.014247521s" totalInitContainerRuntime="70.031743ms" isStatefulPod=true podCreationTimestamp="2026-09-11 13:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:39.829508346 +0000 UTC m=+130.167083934" watchObservedRunningTime="2026-09-11 13:59:39.829610718 +0000 UTC m=+130.167186279"
Sep 11 13:59:39.848382 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:39.848306    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-hbcz5" podStartSLOduration=1.521448913 podStartE2EDuration="2.848289335s" totalImagesPullingTime="1.326840422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:39.847890427 +0000 UTC m=+130.185465988" watchObservedRunningTime="2026-09-11 13:59:39.848289335 +0000 UTC m=+130.185864897"
Sep 11 13:59:40.822234 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:40.822148    2564 generic.go:356] "Generic (PLEG): container finished" podID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerID="460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c" exitCode=0
Sep 11 13:59:40.822673 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:40.822243    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerDied","Data":"460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c"}
Sep 11 13:59:41.324352 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.324281    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cc49cfb9-bn8wj"]
Sep 11 13:59:41.327356 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.327338    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.330546 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.330512    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-242xz\""
Sep 11 13:59:41.330668 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.330552    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 13:59:41.330779 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.330758    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 13:59:41.330983 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.330966    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 13:59:41.330983 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.330981    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 13:59:41.331094 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.331078    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 13:59:41.331134 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.331125    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 13:59:41.331175 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.331142    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 13:59:41.346216 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.346191    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cc49cfb9-bn8wj"]
Sep 11 13:59:41.380211 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.380175    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-console-config\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.380211 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.380214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-serving-cert\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.380357 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.380236    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r65kk\" (UniqueName: \"kubernetes.io/projected/627d2f08-b169-4f11-91bc-8f7806569b6c-kube-api-access-r65kk\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.380357 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.380257    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-oauth-config\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.380357 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.380321    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-service-ca\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.380357 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.380351    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-oauth-serving-cert\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.481019 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.480989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-console-config\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.481019 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.481019    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-serving-cert\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.481143 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.481037    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r65kk\" (UniqueName: \"kubernetes.io/projected/627d2f08-b169-4f11-91bc-8f7806569b6c-kube-api-access-r65kk\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.481143 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.481056    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-oauth-config\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.481243 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.481175    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-service-ca\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.481303 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.481237    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-oauth-serving-cert\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.481887 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.481856    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-console-config\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.481985 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.481915    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-oauth-serving-cert\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.481985 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.481918    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-service-ca\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.483540 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.483518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-oauth-config\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.483697 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.483681    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-serving-cert\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.492108 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.492088    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r65kk\" (UniqueName: \"kubernetes.io/projected/627d2f08-b169-4f11-91bc-8f7806569b6c-kube-api-access-r65kk\") pod \"console-6cc49cfb9-bn8wj\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") " pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.636598 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.636504    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:41.828648 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.828591    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sjsjk" event={"ID":"73e95838-8a95-4047-b739-0f740d140756","Type":"ContainerStarted","Data":"60ae3c3a9a2888970a03e59666e9ee1377e2c5fff728d58065a27e9ab8be93c7"}
Sep 11 13:59:41.829037 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.828674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sjsjk" event={"ID":"73e95838-8a95-4047-b739-0f740d140756","Type":"ContainerStarted","Data":"40d7bddf52d1051214708096276e58cec56ae50ef9c5dcd473b62ed4788af9be"}
Sep 11 13:59:41.850599 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.850552    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sjsjk" podStartSLOduration=129.448964266 podStartE2EDuration="2m10.850530135s" totalImagesPullingTime="1.401565869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:41.849274948 +0000 UTC m=+132.186850509" watchObservedRunningTime="2026-09-11 13:59:41.850530135 +0000 UTC m=+132.188105696"
Sep 11 13:59:41.930969 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:41.930882    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cc49cfb9-bn8wj"]
Sep 11 13:59:42.256929 ip-10-0-129-195 kubenswrapper[2564]: W0911 13:59:42.256853    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod627d2f08_b169_4f11_91bc_8f7806569b6c.slice/crio-0a87817be0816e107ca7f9e4c027c3fb43470364617f4d8bccc22d8db434bd38 WatchSource:0}: Error finding container 0a87817be0816e107ca7f9e4c027c3fb43470364617f4d8bccc22d8db434bd38: Status 404 returned error can't find the container with id 0a87817be0816e107ca7f9e4c027c3fb43470364617f4d8bccc22d8db434bd38
Sep 11 13:59:42.834366 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:42.834330    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc49cfb9-bn8wj" event={"ID":"627d2f08-b169-4f11-91bc-8f7806569b6c","Type":"ContainerStarted","Data":"0a87817be0816e107ca7f9e4c027c3fb43470364617f4d8bccc22d8db434bd38"}
Sep 11 13:59:42.837270 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:42.837245    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerStarted","Data":"808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1"}
Sep 11 13:59:42.837370 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:42.837301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerStarted","Data":"7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771"}
Sep 11 13:59:42.837370 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:42.837328    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerStarted","Data":"3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02"}
Sep 11 13:59:42.837370 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:42.837351    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerStarted","Data":"74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce"}
Sep 11 13:59:42.837480 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:42.837374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerStarted","Data":"2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3"}
Sep 11 13:59:43.844940 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:43.844902    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerStarted","Data":"95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f"}
Sep 11 13:59:43.880900 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:43.880847    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.334415523 podStartE2EDuration="5.880829953s" totalImagesPullingTime="3.5296907s" totalInitContainerRuntime="16.72373ms" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:43.878755276 +0000 UTC m=+134.216330836" watchObservedRunningTime="2026-09-11 13:59:43.880829953 +0000 UTC m=+134.218405538"
Sep 11 13:59:45.855542 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:45.855508    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc49cfb9-bn8wj" event={"ID":"627d2f08-b169-4f11-91bc-8f7806569b6c","Type":"ContainerStarted","Data":"b7c3772970ecf8b9487496cc196d4225617205b2a07ee8ed8fc04b3e4aa0caa5"}
Sep 11 13:59:45.875357 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:45.875306    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cc49cfb9-bn8wj" podStartSLOduration=2.027719179 podStartE2EDuration="4.875290431s" totalImagesPullingTime="2.847571252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:59:45.874336374 +0000 UTC m=+136.211911935" watchObservedRunningTime="2026-09-11 13:59:45.875290431 +0000 UTC m=+136.212865991"
Sep 11 13:59:51.075065 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:51.075032    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d89d5975b-jzjxc"]
Sep 11 13:59:51.637538 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:51.637503    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:51.637701 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:51.637569    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:51.642417 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:51.642387    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 13:59:51.882514 ip-10-0-129-195 kubenswrapper[2564]: I0911 13:59:51.882486    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 14:00:00.882709 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:00.882665    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cc49cfb9-bn8wj"]
Sep 11 14:00:01.915138 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:01.915107    2564 generic.go:356] "Generic (PLEG): container finished" podID="4a8c46b7-5901-4f1e-b98b-9197917b7656" containerID="96422b1ffc5be3374486254870e61ea79aa25c665411c64702786b77fd4b7484" exitCode=0
Sep 11 14:00:01.915522 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:01.915171    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp" event={"ID":"4a8c46b7-5901-4f1e-b98b-9197917b7656","Type":"ContainerDied","Data":"96422b1ffc5be3374486254870e61ea79aa25c665411c64702786b77fd4b7484"}
Sep 11 14:00:01.915522 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:01.915439    2564 scope.go:117] "RemoveContainer" containerID="96422b1ffc5be3374486254870e61ea79aa25c665411c64702786b77fd4b7484"
Sep 11 14:00:02.920818 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:02.920783    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-p58zp" event={"ID":"4a8c46b7-5901-4f1e-b98b-9197917b7656","Type":"ContainerStarted","Data":"82c470107f909d09d65cc4bbf3969d493dc6981df3175a51a7cfabd4044dce04"}
Sep 11 14:00:16.094285 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.094226    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc" podUID="63052ae9-d645-4427-9397-527caeb4823f" containerName="registry" containerID="cri-o://c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f" gracePeriod=30
Sep 11 14:00:16.327372 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.327346    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 14:00:16.459164 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.459073    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63052ae9-d645-4427-9397-527caeb4823f-ca-trust-extracted\") pod \"63052ae9-d645-4427-9397-527caeb4823f\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") "
Sep 11 14:00:16.459164 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.459117    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-image-registry-private-configuration\") pod \"63052ae9-d645-4427-9397-527caeb4823f\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") "
Sep 11 14:00:16.459164 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.459149    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-installation-pull-secrets\") pod \"63052ae9-d645-4427-9397-527caeb4823f\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") "
Sep 11 14:00:16.459422 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.459175    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") pod \"63052ae9-d645-4427-9397-527caeb4823f\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") "
Sep 11 14:00:16.459422 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.459207    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-bound-sa-token\") pod \"63052ae9-d645-4427-9397-527caeb4823f\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") "
Sep 11 14:00:16.459422 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.459231    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-registry-certificates\") pod \"63052ae9-d645-4427-9397-527caeb4823f\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") "
Sep 11 14:00:16.459422 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.459257    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-trusted-ca\") pod \"63052ae9-d645-4427-9397-527caeb4823f\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") "
Sep 11 14:00:16.459422 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.459303    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsx8d\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-kube-api-access-lsx8d\") pod \"63052ae9-d645-4427-9397-527caeb4823f\" (UID: \"63052ae9-d645-4427-9397-527caeb4823f\") "
Sep 11 14:00:16.460202 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.459908    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "63052ae9-d645-4427-9397-527caeb4823f" (UID: "63052ae9-d645-4427-9397-527caeb4823f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:00:16.460202 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.460163    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "63052ae9-d645-4427-9397-527caeb4823f" (UID: "63052ae9-d645-4427-9397-527caeb4823f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:00:16.461874 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.461844    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-kube-api-access-lsx8d" (OuterVolumeSpecName: "kube-api-access-lsx8d") pod "63052ae9-d645-4427-9397-527caeb4823f" (UID: "63052ae9-d645-4427-9397-527caeb4823f"). InnerVolumeSpecName "kube-api-access-lsx8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:16.461874 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.461861    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "63052ae9-d645-4427-9397-527caeb4823f" (UID: "63052ae9-d645-4427-9397-527caeb4823f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:16.462014 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.461894    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "63052ae9-d645-4427-9397-527caeb4823f" (UID: "63052ae9-d645-4427-9397-527caeb4823f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:16.462014 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.461903    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "63052ae9-d645-4427-9397-527caeb4823f" (UID: "63052ae9-d645-4427-9397-527caeb4823f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:16.462014 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.461915    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "63052ae9-d645-4427-9397-527caeb4823f" (UID: "63052ae9-d645-4427-9397-527caeb4823f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:16.467849 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.467824    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63052ae9-d645-4427-9397-527caeb4823f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "63052ae9-d645-4427-9397-527caeb4823f" (UID: "63052ae9-d645-4427-9397-527caeb4823f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:00:16.560012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.559983    2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-installation-pull-secrets\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:16.560012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.560008    2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-registry-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:16.560012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.560018    2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-bound-sa-token\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:16.560191 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.560026    2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-registry-certificates\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:16.560191 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.560034    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/63052ae9-d645-4427-9397-527caeb4823f-trusted-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:16.560191 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.560043    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsx8d\" (UniqueName: \"kubernetes.io/projected/63052ae9-d645-4427-9397-527caeb4823f-kube-api-access-lsx8d\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:16.560191 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.560050    2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/63052ae9-d645-4427-9397-527caeb4823f-ca-trust-extracted\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:16.560191 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.560060    2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/63052ae9-d645-4427-9397-527caeb4823f-image-registry-private-configuration\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:16.978293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.978256    2564 generic.go:356] "Generic (PLEG): container finished" podID="63052ae9-d645-4427-9397-527caeb4823f" containerID="c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f" exitCode=0
Sep 11 14:00:16.978495 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.978299    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc"
Sep 11 14:00:16.978495 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.978334    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc" event={"ID":"63052ae9-d645-4427-9397-527caeb4823f","Type":"ContainerDied","Data":"c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f"}
Sep 11 14:00:16.978495 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.978388    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-d89d5975b-jzjxc" event={"ID":"63052ae9-d645-4427-9397-527caeb4823f","Type":"ContainerDied","Data":"9aca774ca88690fb77375fd4b191473c1099ea24b522bd96cdd7b54dbee791c5"}
Sep 11 14:00:16.978495 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.978410    2564 scope.go:117] "RemoveContainer" containerID="c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f"
Sep 11 14:00:16.987082 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.987063    2564 scope.go:117] "RemoveContainer" containerID="c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f"
Sep 11 14:00:16.987362 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:00:16.987342    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f\": container with ID starting with c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f not found: ID does not exist" containerID="c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f"
Sep 11 14:00:16.987433 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:16.987368    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f"} err="failed to get container status \"c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f\": rpc error: code = NotFound desc = could not find container \"c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f\": container with ID starting with c4fa5380f2404c38f68446d21df92f83a3ee4829f4ee9a0195c683cefb6b0c3f not found: ID does not exist"
Sep 11 14:00:17.007037 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:17.007013    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d89d5975b-jzjxc"]
Sep 11 14:00:17.016544 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:17.016521    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d89d5975b-jzjxc"]
Sep 11 14:00:18.274310 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:18.274269    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63052ae9-d645-4427-9397-527caeb4823f" path="/var/lib/kubelet/pods/63052ae9-d645-4427-9397-527caeb4823f/volumes"
Sep 11 14:00:19.994344 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:19.992773    2564 generic.go:356] "Generic (PLEG): container finished" podID="bcaf3c33-d28c-435e-9901-c977c195aa0e" containerID="ac5cbdc1ab9ff5d1205661607d32fb9a12aff20e9ee342e64e1c20393c966dc9" exitCode=0
Sep 11 14:00:19.994344 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:19.992836    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5" event={"ID":"bcaf3c33-d28c-435e-9901-c977c195aa0e","Type":"ContainerDied","Data":"ac5cbdc1ab9ff5d1205661607d32fb9a12aff20e9ee342e64e1c20393c966dc9"}
Sep 11 14:00:19.994344 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:19.993236    2564 scope.go:117] "RemoveContainer" containerID="ac5cbdc1ab9ff5d1205661607d32fb9a12aff20e9ee342e64e1c20393c966dc9"
Sep 11 14:00:21.001472 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:21.001435    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-xphn5" event={"ID":"bcaf3c33-d28c-435e-9901-c977c195aa0e","Type":"ContainerStarted","Data":"22c3f8b1f4eb60474665f0a49874fca6266cff4018b3d4c743dcca4eeb8858d1"}
Sep 11 14:00:25.907241 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:25.907132    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6cc49cfb9-bn8wj" podUID="627d2f08-b169-4f11-91bc-8f7806569b6c" containerName="console" containerID="cri-o://b7c3772970ecf8b9487496cc196d4225617205b2a07ee8ed8fc04b3e4aa0caa5" gracePeriod=15
Sep 11 14:00:26.023005 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.022977    2564 generic.go:356] "Generic (PLEG): container finished" podID="627d2f08-b169-4f11-91bc-8f7806569b6c" containerID="b7c3772970ecf8b9487496cc196d4225617205b2a07ee8ed8fc04b3e4aa0caa5" exitCode=0
Sep 11 14:00:26.023121 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.023046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc49cfb9-bn8wj" event={"ID":"627d2f08-b169-4f11-91bc-8f7806569b6c","Type":"ContainerDied","Data":"b7c3772970ecf8b9487496cc196d4225617205b2a07ee8ed8fc04b3e4aa0caa5"}
Sep 11 14:00:26.139907 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.139882    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 14:00:26.236668 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.236583    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-oauth-config\") pod \"627d2f08-b169-4f11-91bc-8f7806569b6c\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") "
Sep 11 14:00:26.236800 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.236683    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r65kk\" (UniqueName: \"kubernetes.io/projected/627d2f08-b169-4f11-91bc-8f7806569b6c-kube-api-access-r65kk\") pod \"627d2f08-b169-4f11-91bc-8f7806569b6c\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") "
Sep 11 14:00:26.236800 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.236714    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-console-config\") pod \"627d2f08-b169-4f11-91bc-8f7806569b6c\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") "
Sep 11 14:00:26.236800 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.236735    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-serving-cert\") pod \"627d2f08-b169-4f11-91bc-8f7806569b6c\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") "
Sep 11 14:00:26.236800 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.236757    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-service-ca\") pod \"627d2f08-b169-4f11-91bc-8f7806569b6c\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") "
Sep 11 14:00:26.236800 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.236797    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-oauth-serving-cert\") pod \"627d2f08-b169-4f11-91bc-8f7806569b6c\" (UID: \"627d2f08-b169-4f11-91bc-8f7806569b6c\") "
Sep 11 14:00:26.237143 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.237079    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-console-config" (OuterVolumeSpecName: "console-config") pod "627d2f08-b169-4f11-91bc-8f7806569b6c" (UID: "627d2f08-b169-4f11-91bc-8f7806569b6c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:00:26.237264 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.237169    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-service-ca" (OuterVolumeSpecName: "service-ca") pod "627d2f08-b169-4f11-91bc-8f7806569b6c" (UID: "627d2f08-b169-4f11-91bc-8f7806569b6c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:00:26.237264 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.237245    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "627d2f08-b169-4f11-91bc-8f7806569b6c" (UID: "627d2f08-b169-4f11-91bc-8f7806569b6c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:00:26.238921 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.238899    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/627d2f08-b169-4f11-91bc-8f7806569b6c-kube-api-access-r65kk" (OuterVolumeSpecName: "kube-api-access-r65kk") pod "627d2f08-b169-4f11-91bc-8f7806569b6c" (UID: "627d2f08-b169-4f11-91bc-8f7806569b6c"). InnerVolumeSpecName "kube-api-access-r65kk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:26.239280 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.239261    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "627d2f08-b169-4f11-91bc-8f7806569b6c" (UID: "627d2f08-b169-4f11-91bc-8f7806569b6c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:26.239325 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.239283    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "627d2f08-b169-4f11-91bc-8f7806569b6c" (UID: "627d2f08-b169-4f11-91bc-8f7806569b6c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:26.337444 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.337418    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:26.337444 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.337441    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-oauth-serving-cert\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:26.337599 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.337451    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-oauth-config\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:26.337599 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.337460    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r65kk\" (UniqueName: \"kubernetes.io/projected/627d2f08-b169-4f11-91bc-8f7806569b6c-kube-api-access-r65kk\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:26.337599 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.337468    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/627d2f08-b169-4f11-91bc-8f7806569b6c-console-config\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:26.337599 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:26.337476    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/627d2f08-b169-4f11-91bc-8f7806569b6c-console-serving-cert\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:27.027916 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:27.027887    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cc49cfb9-bn8wj"
Sep 11 14:00:27.028307 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:27.027885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cc49cfb9-bn8wj" event={"ID":"627d2f08-b169-4f11-91bc-8f7806569b6c","Type":"ContainerDied","Data":"0a87817be0816e107ca7f9e4c027c3fb43470364617f4d8bccc22d8db434bd38"}
Sep 11 14:00:27.028307 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:27.028020    2564 scope.go:117] "RemoveContainer" containerID="b7c3772970ecf8b9487496cc196d4225617205b2a07ee8ed8fc04b3e4aa0caa5"
Sep 11 14:00:27.048736 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:27.048673    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cc49cfb9-bn8wj"]
Sep 11 14:00:27.049785 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:27.049765    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cc49cfb9-bn8wj"]
Sep 11 14:00:28.274060 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:28.274027    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="627d2f08-b169-4f11-91bc-8f7806569b6c" path="/var/lib/kubelet/pods/627d2f08-b169-4f11-91bc-8f7806569b6c/volumes"
Sep 11 14:00:57.426971 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:57.426933    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 14:00:57.427465 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:57.427226    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="alertmanager" containerID="cri-o://2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3" gracePeriod=120
Sep 11 14:00:57.427465 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:57.427303    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy-metric" containerID="cri-o://808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1" gracePeriod=120
Sep 11 14:00:57.427465 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:57.427337    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy-web" containerID="cri-o://3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02" gracePeriod=120
Sep 11 14:00:57.428093 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:57.427369    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy" containerID="cri-o://7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771" gracePeriod=120
Sep 11 14:00:57.428093 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:57.427356    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="config-reloader" containerID="cri-o://74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce" gracePeriod=120
Sep 11 14:00:57.428093 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:57.427323    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="prom-label-proxy" containerID="cri-o://95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f" gracePeriod=120
Sep 11 14:00:58.145656 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.145606    2564 generic.go:356] "Generic (PLEG): container finished" podID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerID="95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f" exitCode=0
Sep 11 14:00:58.145656 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.145650    2564 generic.go:356] "Generic (PLEG): container finished" podID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerID="7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771" exitCode=0
Sep 11 14:00:58.145656 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.145659    2564 generic.go:356] "Generic (PLEG): container finished" podID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerID="74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce" exitCode=0
Sep 11 14:00:58.145880 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.145668    2564 generic.go:356] "Generic (PLEG): container finished" podID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerID="2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3" exitCode=0
Sep 11 14:00:58.145880 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.145674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerDied","Data":"95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f"}
Sep 11 14:00:58.145880 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.145737    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerDied","Data":"7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771"}
Sep 11 14:00:58.145880 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.145764    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerDied","Data":"74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce"}
Sep 11 14:00:58.145880 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.145789    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerDied","Data":"2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3"}
Sep 11 14:00:58.672096 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.672073    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:58.759731 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759654    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6c2x\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-kube-api-access-f6c2x\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759731 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759691    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-web\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759731 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759715    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759990 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759747    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-trusted-ca-bundle\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759990 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759784    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-cluster-tls-config\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759990 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759819    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-main-tls\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759990 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759849    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-web-config\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759990 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759876    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-metrics-client-ca\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759990 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759909    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759990 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759941    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-main-db\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.759990 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759964    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-tls-assets\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.760387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.759993    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-out\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.760387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.760028    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-volume\") pod \"fe9051ee-2d26-44e8-87cf-774b18faa44b\" (UID: \"fe9051ee-2d26-44e8-87cf-774b18faa44b\") "
Sep 11 14:00:58.760387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.760161    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:00:58.760387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.760258    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.760584 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.760524    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:00:58.761107 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.761079    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:00:58.762479 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.762447    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:58.762748 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.762706    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:58.762963 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.762935    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:58.763358 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.763332    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-kube-api-access-f6c2x" (OuterVolumeSpecName: "kube-api-access-f6c2x") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "kube-api-access-f6c2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:58.764272 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.764236    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-volume" (OuterVolumeSpecName: "config-volume") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:58.764450 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.764416    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:58.764723 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.764702    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-out" (OuterVolumeSpecName: "config-out") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 14:00:58.765660 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.765613    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:00:58.767716 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.767681    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:58.774006 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.773985    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-web-config" (OuterVolumeSpecName: "web-config") pod "fe9051ee-2d26-44e8-87cf-774b18faa44b" (UID: "fe9051ee-2d26-44e8-87cf-774b18faa44b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:00:58.861190 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861157    2564 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-cluster-tls-config\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861190 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861188    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-main-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861190 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861197    2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-web-config\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861206    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fe9051ee-2d26-44e8-87cf-774b18faa44b-metrics-client-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861216    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861225    2564 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-alertmanager-main-db\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861233    2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-tls-assets\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861241    2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-out\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861249    2564 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861257    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6c2x\" (UniqueName: \"kubernetes.io/projected/fe9051ee-2d26-44e8-87cf-774b18faa44b-kube-api-access-f6c2x\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861265    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:58.861347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:58.861273    2564 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/fe9051ee-2d26-44e8-87cf-774b18faa44b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:00:59.152754 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.152726    2564 generic.go:356] "Generic (PLEG): container finished" podID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerID="808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1" exitCode=0
Sep 11 14:00:59.152754 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.152750    2564 generic.go:356] "Generic (PLEG): container finished" podID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerID="3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02" exitCode=0
Sep 11 14:00:59.152977 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.152796    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.152977 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.152802    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerDied","Data":"808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1"}
Sep 11 14:00:59.152977 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.152885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerDied","Data":"3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02"}
Sep 11 14:00:59.152977 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.152927    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"fe9051ee-2d26-44e8-87cf-774b18faa44b","Type":"ContainerDied","Data":"2aabc53b2007b043b71c62db2930dff4de7888a57d4be6cbaaa85843e554c6d6"}
Sep 11 14:00:59.152977 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.152948    2564 scope.go:117] "RemoveContainer" containerID="95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f"
Sep 11 14:00:59.161425 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.161409    2564 scope.go:117] "RemoveContainer" containerID="808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1"
Sep 11 14:00:59.168808 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.168791    2564 scope.go:117] "RemoveContainer" containerID="7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771"
Sep 11 14:00:59.174965 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.174947    2564 scope.go:117] "RemoveContainer" containerID="3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02"
Sep 11 14:00:59.179967 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.179943    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 14:00:59.181954 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.181931    2564 scope.go:117] "RemoveContainer" containerID="74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce"
Sep 11 14:00:59.185404 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.185381    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 14:00:59.189349 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.189333    2564 scope.go:117] "RemoveContainer" containerID="2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3"
Sep 11 14:00:59.195874 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.195858    2564 scope.go:117] "RemoveContainer" containerID="460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c"
Sep 11 14:00:59.202327 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.202310    2564 scope.go:117] "RemoveContainer" containerID="95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f"
Sep 11 14:00:59.202567 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:00:59.202550    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f\": container with ID starting with 95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f not found: ID does not exist" containerID="95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f"
Sep 11 14:00:59.202612 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.202576    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f"} err="failed to get container status \"95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f\": rpc error: code = NotFound desc = could not find container \"95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f\": container with ID starting with 95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f not found: ID does not exist"
Sep 11 14:00:59.202612 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.202593    2564 scope.go:117] "RemoveContainer" containerID="808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1"
Sep 11 14:00:59.202842 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:00:59.202824    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1\": container with ID starting with 808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1 not found: ID does not exist" containerID="808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1"
Sep 11 14:00:59.202905 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.202850    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1"} err="failed to get container status \"808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1\": rpc error: code = NotFound desc = could not find container \"808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1\": container with ID starting with 808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1 not found: ID does not exist"
Sep 11 14:00:59.202905 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.202873    2564 scope.go:117] "RemoveContainer" containerID="7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771"
Sep 11 14:00:59.203106 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:00:59.203090    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771\": container with ID starting with 7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771 not found: ID does not exist" containerID="7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771"
Sep 11 14:00:59.203146 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.203110    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771"} err="failed to get container status \"7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771\": rpc error: code = NotFound desc = could not find container \"7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771\": container with ID starting with 7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771 not found: ID does not exist"
Sep 11 14:00:59.203146 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.203122    2564 scope.go:117] "RemoveContainer" containerID="3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02"
Sep 11 14:00:59.203334 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:00:59.203319    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02\": container with ID starting with 3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02 not found: ID does not exist" containerID="3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02"
Sep 11 14:00:59.203371 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.203337    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02"} err="failed to get container status \"3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02\": rpc error: code = NotFound desc = could not find container \"3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02\": container with ID starting with 3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02 not found: ID does not exist"
Sep 11 14:00:59.203371 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.203348    2564 scope.go:117] "RemoveContainer" containerID="74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce"
Sep 11 14:00:59.203539 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:00:59.203523    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce\": container with ID starting with 74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce not found: ID does not exist" containerID="74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce"
Sep 11 14:00:59.203577 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.203541    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce"} err="failed to get container status \"74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce\": rpc error: code = NotFound desc = could not find container \"74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce\": container with ID starting with 74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce not found: ID does not exist"
Sep 11 14:00:59.203577 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.203552    2564 scope.go:117] "RemoveContainer" containerID="2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3"
Sep 11 14:00:59.203764 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:00:59.203749    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3\": container with ID starting with 2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3 not found: ID does not exist" containerID="2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3"
Sep 11 14:00:59.203833 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.203771    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3"} err="failed to get container status \"2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3\": rpc error: code = NotFound desc = could not find container \"2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3\": container with ID starting with 2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3 not found: ID does not exist"
Sep 11 14:00:59.203833 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.203789    2564 scope.go:117] "RemoveContainer" containerID="460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c"
Sep 11 14:00:59.204042 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:00:59.204027    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c\": container with ID starting with 460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c not found: ID does not exist" containerID="460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c"
Sep 11 14:00:59.204101 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204048    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c"} err="failed to get container status \"460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c\": rpc error: code = NotFound desc = could not find container \"460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c\": container with ID starting with 460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c not found: ID does not exist"
Sep 11 14:00:59.204101 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204068    2564 scope.go:117] "RemoveContainer" containerID="95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f"
Sep 11 14:00:59.204295 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204277    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f"} err="failed to get container status \"95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f\": rpc error: code = NotFound desc = could not find container \"95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f\": container with ID starting with 95390bc7b33b7586c4fc2255e03d9731d3fcf546d454034d0e36522bdd66122f not found: ID does not exist"
Sep 11 14:00:59.204331 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204303    2564 scope.go:117] "RemoveContainer" containerID="808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1"
Sep 11 14:00:59.204500 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204481    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1"} err="failed to get container status \"808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1\": rpc error: code = NotFound desc = could not find container \"808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1\": container with ID starting with 808d527e5ba5ed836fa21f0bbf900aacc5e564a8560a829d4f0ecaa397a4d4a1 not found: ID does not exist"
Sep 11 14:00:59.204570 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204501    2564 scope.go:117] "RemoveContainer" containerID="7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771"
Sep 11 14:00:59.204722 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204705    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771"} err="failed to get container status \"7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771\": rpc error: code = NotFound desc = could not find container \"7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771\": container with ID starting with 7e084549b3fe22187ace7fe5694e9d73cb1a86be9b9af85e33c547d1d86e2771 not found: ID does not exist"
Sep 11 14:00:59.204765 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204723    2564 scope.go:117] "RemoveContainer" containerID="3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02"
Sep 11 14:00:59.204960 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204941    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02"} err="failed to get container status \"3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02\": rpc error: code = NotFound desc = could not find container \"3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02\": container with ID starting with 3b0fa6e8c121d908ead69f189265322360ff550053a5109326374b871c24bc02 not found: ID does not exist"
Sep 11 14:00:59.205006 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.204961    2564 scope.go:117] "RemoveContainer" containerID="74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce"
Sep 11 14:00:59.205154 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.205139    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce"} err="failed to get container status \"74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce\": rpc error: code = NotFound desc = could not find container \"74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce\": container with ID starting with 74b716086e80cc2496787d8c4663754bb7939670bfa6ee88b3fa4d3d933c93ce not found: ID does not exist"
Sep 11 14:00:59.205193 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.205154    2564 scope.go:117] "RemoveContainer" containerID="2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3"
Sep 11 14:00:59.205337 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.205322    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3"} err="failed to get container status \"2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3\": rpc error: code = NotFound desc = could not find container \"2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3\": container with ID starting with 2875d18fd93d93d0ae5e3e7872595a5759e2dcbed277c7b7c718985cec4fa1d3 not found: ID does not exist"
Sep 11 14:00:59.205378 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.205338    2564 scope.go:117] "RemoveContainer" containerID="460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c"
Sep 11 14:00:59.205517 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.205500    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c"} err="failed to get container status \"460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c\": rpc error: code = NotFound desc = could not find container \"460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c\": container with ID starting with 460c07e3524ebfc8d36447bd1070e0de82cda81654afab784712002acf09cd4c not found: ID does not exist"
Sep 11 14:00:59.222654 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222615    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 14:00:59.222869 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222858    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63052ae9-d645-4427-9397-527caeb4823f" containerName="registry"
Sep 11 14:00:59.222909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222871    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="63052ae9-d645-4427-9397-527caeb4823f" containerName="registry"
Sep 11 14:00:59.222909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222879    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy"
Sep 11 14:00:59.222909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222885    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy"
Sep 11 14:00:59.222909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222893    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy-web"
Sep 11 14:00:59.222909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222899    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy-web"
Sep 11 14:00:59.222909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222904    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy-metric"
Sep 11 14:00:59.222909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222910    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy-metric"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222916    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="init-config-reloader"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222922    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="init-config-reloader"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222928    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="config-reloader"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222934    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="config-reloader"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222941    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="alertmanager"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222946    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="alertmanager"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222952    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="prom-label-proxy"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222957    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="prom-label-proxy"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222962    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="627d2f08-b169-4f11-91bc-8f7806569b6c" containerName="console"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.222968    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="627d2f08-b169-4f11-91bc-8f7806569b6c" containerName="console"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.223008    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="alertmanager"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.223014    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="config-reloader"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.223021    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy-metric"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.223026    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="63052ae9-d645-4427-9397-527caeb4823f" containerName="registry"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.223031    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="prom-label-proxy"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.223037    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="627d2f08-b169-4f11-91bc-8f7806569b6c" containerName="console"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.223042    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy-web"
Sep 11 14:00:59.223104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.223046    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" containerName="kube-rbac-proxy"
Sep 11 14:00:59.227681 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.227663    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.233130 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.233111    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 14:00:59.233569 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.233548    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 14:00:59.233706 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.233689    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 14:00:59.233812 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.233797    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 14:00:59.234012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.233988    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 14:00:59.234107 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.234038    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 14:00:59.234107 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.234093    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sgqtp\""
Sep 11 14:00:59.234197 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.234131    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 14:00:59.239696 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.239681    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 14:00:59.248914 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.248071    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 14:00:59.255293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.255275    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 14:00:59.263135 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-config-out\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263219 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263147    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263219 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263219 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263187    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263377 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263377 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263276    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263377 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263307    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263377 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263377 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263362    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263541 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263390    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263541 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263464    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpnb2\" (UniqueName: \"kubernetes.io/projected/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-kube-api-access-dpnb2\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263541 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263490    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.263541 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.263507    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-web-config\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364491 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364463    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpnb2\" (UniqueName: \"kubernetes.io/projected/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-kube-api-access-dpnb2\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364673 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364507    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364673 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-web-config\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364673 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364661    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-config-out\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364847 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364847 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364847 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364754    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364847 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364847 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364784    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364847 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364847 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364827    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.364847 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364850    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.365213 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.365213 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.364953    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.365883 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.365855    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.366478 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.366454    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.367529 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.367503    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-config-out\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.367622 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.367580    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.367926 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.367820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.367926 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.367820    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.368086 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.368068    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-web-config\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.368205 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.368189    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.368614 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.368592    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.368969 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.368952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-tls-assets\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.369676 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.369660    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-config-volume\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.373036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.373020    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpnb2\" (UniqueName: \"kubernetes.io/projected/46703f2e-56d2-4601-b9b3-f1b634d6a0ed-kube-api-access-dpnb2\") pod \"alertmanager-main-0\" (UID: \"46703f2e-56d2-4601-b9b3-f1b634d6a0ed\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.536021 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.535994    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 14:00:59.820423 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:00:59.820400    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46703f2e_56d2_4601_b9b3_f1b634d6a0ed.slice/crio-c89fbc4b44e77941770e3b56c5e17b5925c58466f68a229c3da4858a97c0f803 WatchSource:0}: Error finding container c89fbc4b44e77941770e3b56c5e17b5925c58466f68a229c3da4858a97c0f803: Status 404 returned error can't find the container with id c89fbc4b44e77941770e3b56c5e17b5925c58466f68a229c3da4858a97c0f803
Sep 11 14:00:59.820751 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:00:59.820433    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 14:01:00.158720 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:00.158621    2564 generic.go:356] "Generic (PLEG): container finished" podID="46703f2e-56d2-4601-b9b3-f1b634d6a0ed" containerID="8a46b3cb5dc97f4d07ffdadc7505a27339d33318846b4f508c4e2c8432b8f34b" exitCode=0
Sep 11 14:01:00.158879 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:00.158721    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46703f2e-56d2-4601-b9b3-f1b634d6a0ed","Type":"ContainerDied","Data":"8a46b3cb5dc97f4d07ffdadc7505a27339d33318846b4f508c4e2c8432b8f34b"}
Sep 11 14:01:00.158879 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:00.158796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46703f2e-56d2-4601-b9b3-f1b634d6a0ed","Type":"ContainerStarted","Data":"c89fbc4b44e77941770e3b56c5e17b5925c58466f68a229c3da4858a97c0f803"}
Sep 11 14:01:00.275711 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:00.275669    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe9051ee-2d26-44e8-87cf-774b18faa44b" path="/var/lib/kubelet/pods/fe9051ee-2d26-44e8-87cf-774b18faa44b/volumes"
Sep 11 14:01:01.166328 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.166295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46703f2e-56d2-4601-b9b3-f1b634d6a0ed","Type":"ContainerStarted","Data":"418a7d442edfaaeb54f0ee712a8fc5fcd7773ae5233a388f6d23587f0ab99c55"}
Sep 11 14:01:01.166736 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.166360    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46703f2e-56d2-4601-b9b3-f1b634d6a0ed","Type":"ContainerStarted","Data":"8dd677aa4a83f6b1b61d88f00636a0f8ad1f8f078cbc8dbeb6c583b082eca11b"}
Sep 11 14:01:01.166736 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.166385    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46703f2e-56d2-4601-b9b3-f1b634d6a0ed","Type":"ContainerStarted","Data":"0ad9aeeee34caf3cb20364ccf105d96ddc736f52b3b4ba3fec166400f822b9d2"}
Sep 11 14:01:01.166736 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.166409    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46703f2e-56d2-4601-b9b3-f1b634d6a0ed","Type":"ContainerStarted","Data":"b5c2d45844c3aa25db12f908ccad47013dfee70147dd46d3beb26f407e87ef19"}
Sep 11 14:01:01.166898 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.166790    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46703f2e-56d2-4601-b9b3-f1b634d6a0ed","Type":"ContainerStarted","Data":"3b52eb5db07abcae98346a6970ddb01af8e4dd0a38c58d8890b96cf5d013646b"}
Sep 11 14:01:01.166948 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.166907    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"46703f2e-56d2-4601-b9b3-f1b634d6a0ed","Type":"ContainerStarted","Data":"4ff2413c0840d1a8954868ecc081711c777f75bc85fd5321e362cd1a5c0122e1"}
Sep 11 14:01:01.197328 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.197285    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.181342088 podStartE2EDuration="2.197270489s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.928401ms" isStatefulPod=false podCreationTimestamp="2026-09-11 14:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:01.195545869 +0000 UTC m=+211.533121428" watchObservedRunningTime="2026-09-11 14:01:01.197270489 +0000 UTC m=+211.534846093"
Sep 11 14:01:01.472487 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.472401    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"]
Sep 11 14:01:01.475887 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.475860    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.479012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.478989    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 14:01:01.479437 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.479417    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 14:01:01.479530 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.479501    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 14:01:01.479597 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.479580    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 14:01:01.479938 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.479923    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 14:01:01.480040 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.480024    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-8sb54\""
Sep 11 14:01:01.488239 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.488215    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 14:01:01.499826 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.499799    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"]
Sep 11 14:01:01.581195 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.581168    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-secret-telemeter-client\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.581195 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.581199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l452j\" (UniqueName: \"kubernetes.io/projected/b2fd31ce-a1f8-4be4-98ba-e445464cb531-kube-api-access-l452j\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.581387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.581221    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2fd31ce-a1f8-4be4-98ba-e445464cb531-serving-certs-ca-bundle\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.581387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.581248    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2fd31ce-a1f8-4be4-98ba-e445464cb531-metrics-client-ca\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.581387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.581265    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2fd31ce-a1f8-4be4-98ba-e445464cb531-telemeter-trusted-ca-bundle\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.581387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.581284    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-federate-client-tls\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.581387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.581305    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.581387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.581322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-telemeter-client-tls\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.682160 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.682124    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.682160 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.682162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-telemeter-client-tls\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.682343 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.682204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-secret-telemeter-client\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.682343 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.682222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l452j\" (UniqueName: \"kubernetes.io/projected/b2fd31ce-a1f8-4be4-98ba-e445464cb531-kube-api-access-l452j\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.682343 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.682247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2fd31ce-a1f8-4be4-98ba-e445464cb531-serving-certs-ca-bundle\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.682343 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.682264    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2fd31ce-a1f8-4be4-98ba-e445464cb531-metrics-client-ca\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.682343 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.682279    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2fd31ce-a1f8-4be4-98ba-e445464cb531-telemeter-trusted-ca-bundle\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.682343 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.682299    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-federate-client-tls\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.683080 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.683048    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b2fd31ce-a1f8-4be4-98ba-e445464cb531-metrics-client-ca\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.683201 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.683110    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2fd31ce-a1f8-4be4-98ba-e445464cb531-serving-certs-ca-bundle\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.683313 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.683288    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2fd31ce-a1f8-4be4-98ba-e445464cb531-telemeter-trusted-ca-bundle\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.684799 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.684777    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-secret-telemeter-client\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.684985 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.684966    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.685048 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.684988    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-telemeter-client-tls\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.685048 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.684999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b2fd31ce-a1f8-4be4-98ba-e445464cb531-federate-client-tls\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.696048 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.696030    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l452j\" (UniqueName: \"kubernetes.io/projected/b2fd31ce-a1f8-4be4-98ba-e445464cb531-kube-api-access-l452j\") pod \"telemeter-client-584fd8cb7d-h4jqw\" (UID: \"b2fd31ce-a1f8-4be4-98ba-e445464cb531\") " pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:01.787026 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:01.786888    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"
Sep 11 14:01:02.066234 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:02.066209    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw"]
Sep 11 14:01:02.069060 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:01:02.069030    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2fd31ce_a1f8_4be4_98ba_e445464cb531.slice/crio-8ba6053ea4f70a7b63a67f29f8345f79bbd9a88943fd448cae4d8c5796bc817d WatchSource:0}: Error finding container 8ba6053ea4f70a7b63a67f29f8345f79bbd9a88943fd448cae4d8c5796bc817d: Status 404 returned error can't find the container with id 8ba6053ea4f70a7b63a67f29f8345f79bbd9a88943fd448cae4d8c5796bc817d
Sep 11 14:01:02.171281 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:02.171248    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw" event={"ID":"b2fd31ce-a1f8-4be4-98ba-e445464cb531","Type":"ContainerStarted","Data":"8ba6053ea4f70a7b63a67f29f8345f79bbd9a88943fd448cae4d8c5796bc817d"}
Sep 11 14:01:04.179982 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.179953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw" event={"ID":"b2fd31ce-a1f8-4be4-98ba-e445464cb531","Type":"ContainerStarted","Data":"bed21590b8523f8d81390aa2ad10ff0405e80076324c84d1b4f893005acc93d7"}
Sep 11 14:01:04.180305 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.180007    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw" event={"ID":"b2fd31ce-a1f8-4be4-98ba-e445464cb531","Type":"ContainerStarted","Data":"3dda341d1e6ba4abb6a49dbd34e73cb89a3a035666119f379fdb530543a24f63"}
Sep 11 14:01:04.180305 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.180027    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw" event={"ID":"b2fd31ce-a1f8-4be4-98ba-e445464cb531","Type":"ContainerStarted","Data":"c981ea16c698b9f3100db67e56dd2462d65a595ee2eb058cdb19fcada1005ac1"}
Sep 11 14:01:04.208340 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.208293    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-584fd8cb7d-h4jqw" podStartSLOduration=1.313853389 podStartE2EDuration="3.208275338s" totalImagesPullingTime="1.894421949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:04.206155227 +0000 UTC m=+214.543730787" watchObservedRunningTime="2026-09-11 14:01:04.208275338 +0000 UTC m=+214.545850897"
Sep 11 14:01:04.344338 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.344304    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d4b6fd77b-cs4dv"]
Sep 11 14:01:04.349436 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.348520    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.352799 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.352778    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 14:01:04.353351 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.353332    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 14:01:04.353469 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.353376    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 14:01:04.354586 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.354568    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 14:01:04.355568 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.355550    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 14:01:04.360957 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.360936    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 14:01:04.361073 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.360980    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 14:01:04.361420 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.361405    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-242xz\""
Sep 11 14:01:04.377207 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.377183    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d4b6fd77b-cs4dv"]
Sep 11 14:01:04.377427 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.377410    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 14:01:04.403187 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.403160    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-console-config\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.403283 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.403193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-trusted-ca-bundle\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.403283 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.403211    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-oauth-serving-cert\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.403283 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.403237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-serving-cert\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.403477 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.403334    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srf5h\" (UniqueName: \"kubernetes.io/projected/684a65db-b015-4d63-a591-c45515237031-kube-api-access-srf5h\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.403477 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.403380    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-oauth-config\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.403477 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.403443    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-service-ca\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.504123 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.504100    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srf5h\" (UniqueName: \"kubernetes.io/projected/684a65db-b015-4d63-a591-c45515237031-kube-api-access-srf5h\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.504218 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.504135    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-oauth-config\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.504218 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.504160    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-service-ca\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.504218 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.504198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-console-config\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.504218 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.504214    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-trusted-ca-bundle\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.504418 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.504228    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-oauth-serving-cert\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.504418 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.504246    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-serving-cert\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.504979 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.504959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-console-config\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.505098 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.505061    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-service-ca\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.505136 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.505116    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-oauth-serving-cert\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.505265 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.505250    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-trusted-ca-bundle\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.506596 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.506570    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-oauth-config\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.506751 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.506736    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-serving-cert\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.513975 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.513952    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srf5h\" (UniqueName: \"kubernetes.io/projected/684a65db-b015-4d63-a591-c45515237031-kube-api-access-srf5h\") pod \"console-5d4b6fd77b-cs4dv\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") " pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.658760 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.658690    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:04.939085 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:04.939014    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d4b6fd77b-cs4dv"]
Sep 11 14:01:04.942910 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:01:04.942886    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684a65db_b015_4d63_a591_c45515237031.slice/crio-70e0a26a326a0a98289c1524610be70e74160b69b877d47ea08b891116668d35 WatchSource:0}: Error finding container 70e0a26a326a0a98289c1524610be70e74160b69b877d47ea08b891116668d35: Status 404 returned error can't find the container with id 70e0a26a326a0a98289c1524610be70e74160b69b877d47ea08b891116668d35
Sep 11 14:01:05.186061 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:05.186019    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4b6fd77b-cs4dv" event={"ID":"684a65db-b015-4d63-a591-c45515237031","Type":"ContainerStarted","Data":"7da23ebe8368770f93220246f58bca9826b5329686a0abad0a770afedfe45bc8"}
Sep 11 14:01:05.186536 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:05.186081    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4b6fd77b-cs4dv" event={"ID":"684a65db-b015-4d63-a591-c45515237031","Type":"ContainerStarted","Data":"70e0a26a326a0a98289c1524610be70e74160b69b877d47ea08b891116668d35"}
Sep 11 14:01:05.215496 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:05.215385    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d4b6fd77b-cs4dv" podStartSLOduration=1.215368726 podStartE2EDuration="1.215368726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:01:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:01:05.214459488 +0000 UTC m=+215.552035029" watchObservedRunningTime="2026-09-11 14:01:05.215368726 +0000 UTC m=+215.552944286"
Sep 11 14:01:14.659126 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:14.659089    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:14.659521 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:14.659153    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:14.664148 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:14.664128    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:01:15.229201 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:01:15.229174    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:02:27.419378 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:27.419342    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d4b6fd77b-cs4dv"]
Sep 11 14:02:30.094969 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:30.094939    2564 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 14:02:48.365551 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.365517    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"]
Sep 11 14:02:48.368753 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.368733    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.376762 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.376741    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 14:02:48.376974 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.376958    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 11 14:02:48.381025 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.381008    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 14:02:48.382880 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.382865    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 14:02:48.394177 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.394154    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"]
Sep 11 14:02:48.459866 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.459846    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"]
Sep 11 14:02:48.462992 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.462976    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.465315 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.465289    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 11 14:02:48.465654 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.465621    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 11 14:02:48.465749 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.465674    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 11 14:02:48.466184 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.466168    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 11 14:02:48.480212 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.480189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg4km\" (UniqueName: \"kubernetes.io/projected/89645292-1cf4-449c-b7fe-a1a8f5126134-kube-api-access-kg4km\") pod \"klusterlet-addon-workmgr-8c544546c-8tk8g\" (UID: \"89645292-1cf4-449c-b7fe-a1a8f5126134\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.480288 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.480229    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89645292-1cf4-449c-b7fe-a1a8f5126134-tmp\") pod \"klusterlet-addon-workmgr-8c544546c-8tk8g\" (UID: \"89645292-1cf4-449c-b7fe-a1a8f5126134\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.480341 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.480324    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/89645292-1cf4-449c-b7fe-a1a8f5126134-klusterlet-config\") pod \"klusterlet-addon-workmgr-8c544546c-8tk8g\" (UID: \"89645292-1cf4-449c-b7fe-a1a8f5126134\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.483072 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.483051    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"]
Sep 11 14:02:48.581609 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.581572    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-ca\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.581782 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.581647    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg4km\" (UniqueName: \"kubernetes.io/projected/89645292-1cf4-449c-b7fe-a1a8f5126134-kube-api-access-kg4km\") pod \"klusterlet-addon-workmgr-8c544546c-8tk8g\" (UID: \"89645292-1cf4-449c-b7fe-a1a8f5126134\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.581782 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.581670    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.581782 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.581695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.581895 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.581795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89645292-1cf4-449c-b7fe-a1a8f5126134-tmp\") pod \"klusterlet-addon-workmgr-8c544546c-8tk8g\" (UID: \"89645292-1cf4-449c-b7fe-a1a8f5126134\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.581895 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.581810    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-hub\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.581895 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.581828    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlrpg\" (UniqueName: \"kubernetes.io/projected/2565ca93-06db-439c-80b0-5b04a2a3c492-kube-api-access-rlrpg\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.581895 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.581869    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2565ca93-06db-439c-80b0-5b04a2a3c492-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.582078 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.581930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/89645292-1cf4-449c-b7fe-a1a8f5126134-klusterlet-config\") pod \"klusterlet-addon-workmgr-8c544546c-8tk8g\" (UID: \"89645292-1cf4-449c-b7fe-a1a8f5126134\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.582181 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.582164    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89645292-1cf4-449c-b7fe-a1a8f5126134-tmp\") pod \"klusterlet-addon-workmgr-8c544546c-8tk8g\" (UID: \"89645292-1cf4-449c-b7fe-a1a8f5126134\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.584199 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.584174    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/89645292-1cf4-449c-b7fe-a1a8f5126134-klusterlet-config\") pod \"klusterlet-addon-workmgr-8c544546c-8tk8g\" (UID: \"89645292-1cf4-449c-b7fe-a1a8f5126134\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.590350 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.590328    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg4km\" (UniqueName: \"kubernetes.io/projected/89645292-1cf4-449c-b7fe-a1a8f5126134-kube-api-access-kg4km\") pod \"klusterlet-addon-workmgr-8c544546c-8tk8g\" (UID: \"89645292-1cf4-449c-b7fe-a1a8f5126134\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.678052 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.677977    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:48.683152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.683123    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2565ca93-06db-439c-80b0-5b04a2a3c492-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.683263 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.683163    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-ca\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.683263 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.683186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.683263 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.683244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.683439 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.683296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-hub\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.683439 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.683317    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlrpg\" (UniqueName: \"kubernetes.io/projected/2565ca93-06db-439c-80b0-5b04a2a3c492-kube-api-access-rlrpg\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.684274 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.684245    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2565ca93-06db-439c-80b0-5b04a2a3c492-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.685836 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.685811    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.685916 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.685816    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-ca\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.685992 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.685974    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.686118 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.686098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2565ca93-06db-439c-80b0-5b04a2a3c492-hub\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.691925 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.691904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlrpg\" (UniqueName: \"kubernetes.io/projected/2565ca93-06db-439c-80b0-5b04a2a3c492-kube-api-access-rlrpg\") pod \"cluster-proxy-proxy-agent-7d6d4676c-zffbb\" (UID: \"2565ca93-06db-439c-80b0-5b04a2a3c492\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.784641 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.784595    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"
Sep 11 14:02:48.954747 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.954675    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"]
Sep 11 14:02:48.957587 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:02:48.957561    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89645292_1cf4_449c_b7fe_a1a8f5126134.slice/crio-24922cc72c5ef46f77a9257d6e97429fb8e3e8c266ac5f9b6a738d5d8d9a7bf5 WatchSource:0}: Error finding container 24922cc72c5ef46f77a9257d6e97429fb8e3e8c266ac5f9b6a738d5d8d9a7bf5: Status 404 returned error can't find the container with id 24922cc72c5ef46f77a9257d6e97429fb8e3e8c266ac5f9b6a738d5d8d9a7bf5
Sep 11 14:02:48.959437 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:48.959423    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:02:49.066317 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:49.066292    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb"]
Sep 11 14:02:49.069353 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:02:49.069329    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2565ca93_06db_439c_80b0_5b04a2a3c492.slice/crio-a0dc20f9bdd97dcb7d05108ba6a5fdac7cc8ab12aed28bf9480d85580782561a WatchSource:0}: Error finding container a0dc20f9bdd97dcb7d05108ba6a5fdac7cc8ab12aed28bf9480d85580782561a: Status 404 returned error can't find the container with id a0dc20f9bdd97dcb7d05108ba6a5fdac7cc8ab12aed28bf9480d85580782561a
Sep 11 14:02:49.554274 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:49.554238    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g" event={"ID":"89645292-1cf4-449c-b7fe-a1a8f5126134","Type":"ContainerStarted","Data":"24922cc72c5ef46f77a9257d6e97429fb8e3e8c266ac5f9b6a738d5d8d9a7bf5"}
Sep 11 14:02:49.555422 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:49.555399    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb" event={"ID":"2565ca93-06db-439c-80b0-5b04a2a3c492","Type":"ContainerStarted","Data":"a0dc20f9bdd97dcb7d05108ba6a5fdac7cc8ab12aed28bf9480d85580782561a"}
Sep 11 14:02:52.439136 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:52.439022    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d4b6fd77b-cs4dv" podUID="684a65db-b015-4d63-a591-c45515237031" containerName="console" containerID="cri-o://7da23ebe8368770f93220246f58bca9826b5329686a0abad0a770afedfe45bc8" gracePeriod=15
Sep 11 14:02:53.580712 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:53.580677    2564 generic.go:356] "Generic (PLEG): container finished" podID="684a65db-b015-4d63-a591-c45515237031" containerID="7da23ebe8368770f93220246f58bca9826b5329686a0abad0a770afedfe45bc8" exitCode=0
Sep 11 14:02:53.581152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:53.580755    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4b6fd77b-cs4dv" event={"ID":"684a65db-b015-4d63-a591-c45515237031","Type":"ContainerDied","Data":"7da23ebe8368770f93220246f58bca9826b5329686a0abad0a770afedfe45bc8"}
Sep 11 14:02:54.645248 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.645226    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:02:54.737810 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.737791    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-console-config\") pod \"684a65db-b015-4d63-a591-c45515237031\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") "
Sep 11 14:02:54.737912 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.737825    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-serving-cert\") pod \"684a65db-b015-4d63-a591-c45515237031\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") "
Sep 11 14:02:54.737912 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.737862    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-service-ca\") pod \"684a65db-b015-4d63-a591-c45515237031\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") "
Sep 11 14:02:54.738039 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.737912    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srf5h\" (UniqueName: \"kubernetes.io/projected/684a65db-b015-4d63-a591-c45515237031-kube-api-access-srf5h\") pod \"684a65db-b015-4d63-a591-c45515237031\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") "
Sep 11 14:02:54.738039 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.737952    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-oauth-config\") pod \"684a65db-b015-4d63-a591-c45515237031\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") "
Sep 11 14:02:54.738039 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.738010    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-oauth-serving-cert\") pod \"684a65db-b015-4d63-a591-c45515237031\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") "
Sep 11 14:02:54.738039 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.738035    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-trusted-ca-bundle\") pod \"684a65db-b015-4d63-a591-c45515237031\" (UID: \"684a65db-b015-4d63-a591-c45515237031\") "
Sep 11 14:02:54.738572 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.738263    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-console-config" (OuterVolumeSpecName: "console-config") pod "684a65db-b015-4d63-a591-c45515237031" (UID: "684a65db-b015-4d63-a591-c45515237031"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:02:54.738572 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.738509    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-service-ca" (OuterVolumeSpecName: "service-ca") pod "684a65db-b015-4d63-a591-c45515237031" (UID: "684a65db-b015-4d63-a591-c45515237031"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:02:54.738572 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.738537    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "684a65db-b015-4d63-a591-c45515237031" (UID: "684a65db-b015-4d63-a591-c45515237031"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:02:54.738572 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.738561    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "684a65db-b015-4d63-a591-c45515237031" (UID: "684a65db-b015-4d63-a591-c45515237031"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:02:54.740297 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.740275    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "684a65db-b015-4d63-a591-c45515237031" (UID: "684a65db-b015-4d63-a591-c45515237031"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:02:54.740422 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.740366    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/684a65db-b015-4d63-a591-c45515237031-kube-api-access-srf5h" (OuterVolumeSpecName: "kube-api-access-srf5h") pod "684a65db-b015-4d63-a591-c45515237031" (UID: "684a65db-b015-4d63-a591-c45515237031"). InnerVolumeSpecName "kube-api-access-srf5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:02:54.740422 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.740370    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "684a65db-b015-4d63-a591-c45515237031" (UID: "684a65db-b015-4d63-a591-c45515237031"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:02:54.839151 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.839127    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-console-config\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:02:54.839231 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.839159    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-serving-cert\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:02:54.839231 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.839178    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:02:54.839231 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.839191    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srf5h\" (UniqueName: \"kubernetes.io/projected/684a65db-b015-4d63-a591-c45515237031-kube-api-access-srf5h\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:02:54.839231 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.839204    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/684a65db-b015-4d63-a591-c45515237031-console-oauth-config\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:02:54.839231 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.839217    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-oauth-serving-cert\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:02:54.839231 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:54.839228    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/684a65db-b015-4d63-a591-c45515237031-trusted-ca-bundle\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:02:55.590936 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.590907    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4b6fd77b-cs4dv"
Sep 11 14:02:55.590936 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.590923    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4b6fd77b-cs4dv" event={"ID":"684a65db-b015-4d63-a591-c45515237031","Type":"ContainerDied","Data":"70e0a26a326a0a98289c1524610be70e74160b69b877d47ea08b891116668d35"}
Sep 11 14:02:55.591198 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.590978    2564 scope.go:117] "RemoveContainer" containerID="7da23ebe8368770f93220246f58bca9826b5329686a0abad0a770afedfe45bc8"
Sep 11 14:02:55.592601 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.592574    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g" event={"ID":"89645292-1cf4-449c-b7fe-a1a8f5126134","Type":"ContainerStarted","Data":"1b58619b768af341dc6cf33805333a358ea905c709272fc44206e3b3bcb91371"}
Sep 11 14:02:55.592816 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.592651    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:55.594258 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.594235    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g"
Sep 11 14:02:55.594687 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.594663    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb" event={"ID":"2565ca93-06db-439c-80b0-5b04a2a3c492","Type":"ContainerStarted","Data":"4c2433c0f935276003e0c539eb1b76e5c09c8a8ec8e4c36b6140b3a4c65bbac0"}
Sep 11 14:02:55.594800 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.594699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb" event={"ID":"2565ca93-06db-439c-80b0-5b04a2a3c492","Type":"ContainerStarted","Data":"80dfe449dd20806c3cf79a4f603a59ee3f4d4ee12cca0e9e5351a4139d44e695"}
Sep 11 14:02:55.594800 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.594727    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb" event={"ID":"2565ca93-06db-439c-80b0-5b04a2a3c492","Type":"ContainerStarted","Data":"74cd062f9ed126786dfa9863da2ecbba8d1e5310d8c3165a8a0780b36a8cd436"}
Sep 11 14:02:55.613674 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.613617    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-8c544546c-8tk8g" podStartSLOduration=1.8893892289999998 podStartE2EDuration="7.613604539s" totalImagesPullingTime="5.72421531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:55.612520117 +0000 UTC m=+325.950095678" watchObservedRunningTime="2026-09-11 14:02:55.613604539 +0000 UTC m=+325.951180099"
Sep 11 14:02:55.632249 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.632202    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7d6d4676c-zffbb" podStartSLOduration=2.02238819 podStartE2EDuration="7.632188559s" totalImagesPullingTime="5.609800369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:02:55.631470034 +0000 UTC m=+325.969045595" watchObservedRunningTime="2026-09-11 14:02:55.632188559 +0000 UTC m=+325.969764118"
Sep 11 14:02:55.645386 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.645363    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d4b6fd77b-cs4dv"]
Sep 11 14:02:55.647304 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:55.647283    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d4b6fd77b-cs4dv"]
Sep 11 14:02:56.273937 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:02:56.273895    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="684a65db-b015-4d63-a591-c45515237031" path="/var/lib/kubelet/pods/684a65db-b015-4d63-a591-c45515237031/volumes"
Sep 11 14:05:21.647678 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.647644    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"]
Sep 11 14:05:21.648232 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.648048    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684a65db-b015-4d63-a591-c45515237031" containerName="console"
Sep 11 14:05:21.648232 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.648066    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="684a65db-b015-4d63-a591-c45515237031" containerName="console"
Sep 11 14:05:21.648232 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.648163    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="684a65db-b015-4d63-a591-c45515237031" containerName="console"
Sep 11 14:05:21.650968 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.650949    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"
Sep 11 14:05:21.653521 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.653501    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 14:05:21.654311 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.654293    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 14:05:21.654403 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.654324    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-p2tnw\""
Sep 11 14:05:21.659927 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.659905    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"]
Sep 11 14:05:21.758275 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.758241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0e64d3-36c3-4b71-b133-cf64bed28532-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ttsph\" (UID: \"5e0e64d3-36c3-4b71-b133-cf64bed28532\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"
Sep 11 14:05:21.758453 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.758346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdxjr\" (UniqueName: \"kubernetes.io/projected/5e0e64d3-36c3-4b71-b133-cf64bed28532-kube-api-access-tdxjr\") pod \"openshift-lws-operator-fd8ccff4c-ttsph\" (UID: \"5e0e64d3-36c3-4b71-b133-cf64bed28532\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"
Sep 11 14:05:21.858856 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.858819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdxjr\" (UniqueName: \"kubernetes.io/projected/5e0e64d3-36c3-4b71-b133-cf64bed28532-kube-api-access-tdxjr\") pod \"openshift-lws-operator-fd8ccff4c-ttsph\" (UID: \"5e0e64d3-36c3-4b71-b133-cf64bed28532\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"
Sep 11 14:05:21.859018 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.858871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0e64d3-36c3-4b71-b133-cf64bed28532-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ttsph\" (UID: \"5e0e64d3-36c3-4b71-b133-cf64bed28532\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"
Sep 11 14:05:21.859230 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.859212    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e0e64d3-36c3-4b71-b133-cf64bed28532-tmp\") pod \"openshift-lws-operator-fd8ccff4c-ttsph\" (UID: \"5e0e64d3-36c3-4b71-b133-cf64bed28532\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"
Sep 11 14:05:21.867898 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.867878    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdxjr\" (UniqueName: \"kubernetes.io/projected/5e0e64d3-36c3-4b71-b133-cf64bed28532-kube-api-access-tdxjr\") pod \"openshift-lws-operator-fd8ccff4c-ttsph\" (UID: \"5e0e64d3-36c3-4b71-b133-cf64bed28532\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"
Sep 11 14:05:21.959906 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:21.959819    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"
Sep 11 14:05:22.229586 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:22.229559    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph"]
Sep 11 14:05:22.232044 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:05:22.232019    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e0e64d3_36c3_4b71_b133_cf64bed28532.slice/crio-4f423aa9ea4337ef3320a354372dfd100d9a092d089a5338c898156fed489c2b WatchSource:0}: Error finding container 4f423aa9ea4337ef3320a354372dfd100d9a092d089a5338c898156fed489c2b: Status 404 returned error can't find the container with id 4f423aa9ea4337ef3320a354372dfd100d9a092d089a5338c898156fed489c2b
Sep 11 14:05:23.144133 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:23.144100    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph" event={"ID":"5e0e64d3-36c3-4b71-b133-cf64bed28532","Type":"ContainerStarted","Data":"4f423aa9ea4337ef3320a354372dfd100d9a092d089a5338c898156fed489c2b"}
Sep 11 14:05:28.167742 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:28.167650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph" event={"ID":"5e0e64d3-36c3-4b71-b133-cf64bed28532","Type":"ContainerStarted","Data":"dd38da5ee397be1708eba0fe850ad44509bb627e17930d7fa4c6acde6444ad8a"}
Sep 11 14:05:28.185840 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:28.185790    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-ttsph" podStartSLOduration=1.5019832800000001 podStartE2EDuration="7.18577544s" totalImagesPullingTime="5.68379216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:28.184017043 +0000 UTC m=+478.521592606" watchObservedRunningTime="2026-09-11 14:05:28.18577544 +0000 UTC m=+478.523351000"
Sep 11 14:05:43.603808 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.603777    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-779774756-xjsrj"]
Sep 11 14:05:43.607129 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.607106    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.611488 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.611461    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 14:05:43.611598 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.611502    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jgfjv\""
Sep 11 14:05:43.612373 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.612349    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 14:05:43.612466 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.612392    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 14:05:43.621355 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.621335    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-779774756-xjsrj"]
Sep 11 14:05:43.735742 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.735712    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6548a881-f3ea-43ff-bb08-36a48be1dff6-cert\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.735906 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.735769    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hchq\" (UniqueName: \"kubernetes.io/projected/6548a881-f3ea-43ff-bb08-36a48be1dff6-kube-api-access-8hchq\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.735906 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.735843    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6548a881-f3ea-43ff-bb08-36a48be1dff6-manager-config\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.735906 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.735883    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6548a881-f3ea-43ff-bb08-36a48be1dff6-metrics-cert\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.836978 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.836947    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6548a881-f3ea-43ff-bb08-36a48be1dff6-manager-config\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.836978 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.836983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6548a881-f3ea-43ff-bb08-36a48be1dff6-metrics-cert\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.837167 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.837012    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6548a881-f3ea-43ff-bb08-36a48be1dff6-cert\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.837167 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.837120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hchq\" (UniqueName: \"kubernetes.io/projected/6548a881-f3ea-43ff-bb08-36a48be1dff6-kube-api-access-8hchq\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.837697 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.837675    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6548a881-f3ea-43ff-bb08-36a48be1dff6-manager-config\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.839494 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.839466    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6548a881-f3ea-43ff-bb08-36a48be1dff6-metrics-cert\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.839587 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.839500    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6548a881-f3ea-43ff-bb08-36a48be1dff6-cert\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.845904 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.845884    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hchq\" (UniqueName: \"kubernetes.io/projected/6548a881-f3ea-43ff-bb08-36a48be1dff6-kube-api-access-8hchq\") pod \"lws-controller-manager-779774756-xjsrj\" (UID: \"6548a881-f3ea-43ff-bb08-36a48be1dff6\") " pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:43.916291 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:43.916237    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:44.203842 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:44.203795    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-779774756-xjsrj"]
Sep 11 14:05:44.205967 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:05:44.205937    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6548a881_f3ea_43ff_bb08_36a48be1dff6.slice/crio-5f2caefe444c27be9f9e15370a98c261f13e6534d94f546e958d52df8ada729a WatchSource:0}: Error finding container 5f2caefe444c27be9f9e15370a98c261f13e6534d94f546e958d52df8ada729a: Status 404 returned error can't find the container with id 5f2caefe444c27be9f9e15370a98c261f13e6534d94f546e958d52df8ada729a
Sep 11 14:05:44.230295 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:44.230268    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj" event={"ID":"6548a881-f3ea-43ff-bb08-36a48be1dff6","Type":"ContainerStarted","Data":"5f2caefe444c27be9f9e15370a98c261f13e6534d94f546e958d52df8ada729a"}
Sep 11 14:05:46.822203 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.822170    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"]
Sep 11 14:05:46.825458 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.825437    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:46.828094 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.828041    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 14:05:46.828483 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.828458    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 14:05:46.828789 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.828766    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 14:05:46.828894 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.828842    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 14:05:46.828894 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.828876    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2hc6w\""
Sep 11 14:05:46.842327 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.842299    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"]
Sep 11 14:05:46.962342 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.962310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2db6f35e-1e39-4396-a604-ab0569226528-manifests\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:46.962492 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.962354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2db6f35e-1e39-4396-a604-ab0569226528-webhook-cert\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:46.962492 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.962391    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2db6f35e-1e39-4396-a604-ab0569226528-tmp\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:46.962492 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.962444    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2db6f35e-1e39-4396-a604-ab0569226528-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:46.962597 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:46.962498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7zfw\" (UniqueName: \"kubernetes.io/projected/2db6f35e-1e39-4396-a604-ab0569226528-kube-api-access-g7zfw\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.063647 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.063605    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2db6f35e-1e39-4396-a604-ab0569226528-tmp\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.063782 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.063674    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2db6f35e-1e39-4396-a604-ab0569226528-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.063782 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.063720    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7zfw\" (UniqueName: \"kubernetes.io/projected/2db6f35e-1e39-4396-a604-ab0569226528-kube-api-access-g7zfw\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.063782 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.063745    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2db6f35e-1e39-4396-a604-ab0569226528-manifests\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.063896 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.063792    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2db6f35e-1e39-4396-a604-ab0569226528-webhook-cert\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.064150 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.064123    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2db6f35e-1e39-4396-a604-ab0569226528-manifests\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.066061 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.066035    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2db6f35e-1e39-4396-a604-ab0569226528-tmp\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.066274 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.066255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2db6f35e-1e39-4396-a604-ab0569226528-webhook-cert\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.066316 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.066255    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2db6f35e-1e39-4396-a604-ab0569226528-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.080971 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.080918    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7zfw\" (UniqueName: \"kubernetes.io/projected/2db6f35e-1e39-4396-a604-ab0569226528-kube-api-access-g7zfw\") pod \"opendatahub-operator-controller-manager-5586bcdcf6-f77wt\" (UID: \"2db6f35e-1e39-4396-a604-ab0569226528\") " pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.136947 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.136923    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:47.249426 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.249393    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj" event={"ID":"6548a881-f3ea-43ff-bb08-36a48be1dff6","Type":"ContainerStarted","Data":"d7db025b9acdf01c85b8d72c6b2eb3bb14c64bc09a14447bed49c61116f034a3"}
Sep 11 14:05:47.249583 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.249513    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:05:47.318706 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.318661    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj" podStartSLOduration=1.947727641 podStartE2EDuration="4.318621589s" totalImagesPullingTime="2.370893948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:47.318599292 +0000 UTC m=+497.656174852" watchObservedRunningTime="2026-09-11 14:05:47.318621589 +0000 UTC m=+497.656197149"
Sep 11 14:05:47.441925 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:47.441844    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"]
Sep 11 14:05:47.445784 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:05:47.445758    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db6f35e_1e39_4396_a604_ab0569226528.slice/crio-46c3a336f4da1e40c12d8bb0ba0472eb6adc10960efe3fa4ea136a7e208dfe38 WatchSource:0}: Error finding container 46c3a336f4da1e40c12d8bb0ba0472eb6adc10960efe3fa4ea136a7e208dfe38: Status 404 returned error can't find the container with id 46c3a336f4da1e40c12d8bb0ba0472eb6adc10960efe3fa4ea136a7e208dfe38
Sep 11 14:05:48.256723 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:48.256681    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt" event={"ID":"2db6f35e-1e39-4396-a604-ab0569226528","Type":"ContainerStarted","Data":"46c3a336f4da1e40c12d8bb0ba0472eb6adc10960efe3fa4ea136a7e208dfe38"}
Sep 11 14:05:51.272449 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:51.272416    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt" event={"ID":"2db6f35e-1e39-4396-a604-ab0569226528","Type":"ContainerStarted","Data":"5b0324d7644a4c6adc8b550311c32068e28306ceb1b14ff9137c87706d3e0c9d"}
Sep 11 14:05:52.279886 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:52.279853    2564 generic.go:356] "Generic (PLEG): container finished" podID="2db6f35e-1e39-4396-a604-ab0569226528" containerID="5b0324d7644a4c6adc8b550311c32068e28306ceb1b14ff9137c87706d3e0c9d" exitCode=0
Sep 11 14:05:52.280317 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:52.279900    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt" event={"ID":"2db6f35e-1e39-4396-a604-ab0569226528","Type":"ContainerDied","Data":"5b0324d7644a4c6adc8b550311c32068e28306ceb1b14ff9137c87706d3e0c9d"}
Sep 11 14:05:53.285910 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:53.285869    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt" event={"ID":"2db6f35e-1e39-4396-a604-ab0569226528","Type":"ContainerStarted","Data":"9431b52225b8e8e8413906a8fba72bd9f1a95c919c498f203185e78f637d8729"}
Sep 11 14:05:53.286341 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:53.286012    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:05:53.316135 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:53.316086    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt" podStartSLOduration=2.743795578 podStartE2EDuration="7.316071536s" totalImagesPullingTime="3.571093424s" totalInitContainerRuntime="1.001182534s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:05:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:05:53.314280921 +0000 UTC m=+503.651856483" watchObservedRunningTime="2026-09-11 14:05:53.316071536 +0000 UTC m=+503.653647116"
Sep 11 14:05:58.258779 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:05:58.258750    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-779774756-xjsrj"
Sep 11 14:06:04.291926 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:04.291898    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5586bcdcf6-f77wt"
Sep 11 14:06:17.308491 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.308455    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kzl4k"]
Sep 11 14:06:17.318410 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.318385    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:17.321872 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.321849    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 14:06:17.321986 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.321896    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wbn6v\""
Sep 11 14:06:17.343198 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.343174    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kzl4k"]
Sep 11 14:06:17.390519 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.390493    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c905ec0d-2626-4e38-918d-df403ab57f76-cert\") pod \"odh-model-controller-7998c67dc4-kzl4k\" (UID: \"c905ec0d-2626-4e38-918d-df403ab57f76\") " pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:17.390689 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.390564    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrr8c\" (UniqueName: \"kubernetes.io/projected/c905ec0d-2626-4e38-918d-df403ab57f76-kube-api-access-zrr8c\") pod \"odh-model-controller-7998c67dc4-kzl4k\" (UID: \"c905ec0d-2626-4e38-918d-df403ab57f76\") " pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:17.491364 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.491332    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrr8c\" (UniqueName: \"kubernetes.io/projected/c905ec0d-2626-4e38-918d-df403ab57f76-kube-api-access-zrr8c\") pod \"odh-model-controller-7998c67dc4-kzl4k\" (UID: \"c905ec0d-2626-4e38-918d-df403ab57f76\") " pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:17.491511 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.491370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c905ec0d-2626-4e38-918d-df403ab57f76-cert\") pod \"odh-model-controller-7998c67dc4-kzl4k\" (UID: \"c905ec0d-2626-4e38-918d-df403ab57f76\") " pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:17.493644 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.493608    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c905ec0d-2626-4e38-918d-df403ab57f76-cert\") pod \"odh-model-controller-7998c67dc4-kzl4k\" (UID: \"c905ec0d-2626-4e38-918d-df403ab57f76\") " pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:17.506667 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.506645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrr8c\" (UniqueName: \"kubernetes.io/projected/c905ec0d-2626-4e38-918d-df403ab57f76-kube-api-access-zrr8c\") pod \"odh-model-controller-7998c67dc4-kzl4k\" (UID: \"c905ec0d-2626-4e38-918d-df403ab57f76\") " pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:17.627876 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.627804    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:17.898715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.898607    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kzl4k"]
Sep 11 14:06:17.901425 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:06:17.901389    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc905ec0d_2626_4e38_918d_df403ab57f76.slice/crio-3774d7367649c65999d903ce0663a1ed21c03ec9fa7fee248cd90dc3b34d671e WatchSource:0}: Error finding container 3774d7367649c65999d903ce0663a1ed21c03ec9fa7fee248cd90dc3b34d671e: Status 404 returned error can't find the container with id 3774d7367649c65999d903ce0663a1ed21c03ec9fa7fee248cd90dc3b34d671e
Sep 11 14:06:17.954348 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.954320    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cx8rc"]
Sep 11 14:06:17.959081 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.959059    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:17.963882 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.963857    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-8jl26\""
Sep 11 14:06:17.964101 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.964086    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 11 14:06:17.972017 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.971993    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cx8rc"]
Sep 11 14:06:17.997243 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.997213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d07caad-64bc-46d3-9b3d-96a7c3c69e24-tls-certs\") pod \"model-serving-api-54d89bd5fc-cx8rc\" (UID: \"5d07caad-64bc-46d3-9b3d-96a7c3c69e24\") " pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:17.997367 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:17.997253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s2l4\" (UniqueName: \"kubernetes.io/projected/5d07caad-64bc-46d3-9b3d-96a7c3c69e24-kube-api-access-9s2l4\") pod \"model-serving-api-54d89bd5fc-cx8rc\" (UID: \"5d07caad-64bc-46d3-9b3d-96a7c3c69e24\") " pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:18.098139 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:18.098107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d07caad-64bc-46d3-9b3d-96a7c3c69e24-tls-certs\") pod \"model-serving-api-54d89bd5fc-cx8rc\" (UID: \"5d07caad-64bc-46d3-9b3d-96a7c3c69e24\") " pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:18.098269 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:18.098147    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s2l4\" (UniqueName: \"kubernetes.io/projected/5d07caad-64bc-46d3-9b3d-96a7c3c69e24-kube-api-access-9s2l4\") pod \"model-serving-api-54d89bd5fc-cx8rc\" (UID: \"5d07caad-64bc-46d3-9b3d-96a7c3c69e24\") " pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:18.098314 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:06:18.098263    2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 11 14:06:18.098348 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:06:18.098328    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d07caad-64bc-46d3-9b3d-96a7c3c69e24-tls-certs podName:5d07caad-64bc-46d3-9b3d-96a7c3c69e24 nodeName:}" failed. No retries permitted until 2026-09-11 14:06:18.598309631 +0000 UTC m=+528.935885183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/5d07caad-64bc-46d3-9b3d-96a7c3c69e24-tls-certs") pod "model-serving-api-54d89bd5fc-cx8rc" (UID: "5d07caad-64bc-46d3-9b3d-96a7c3c69e24") : secret "model-serving-api-tls" not found
Sep 11 14:06:18.106652 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:18.106616    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s2l4\" (UniqueName: \"kubernetes.io/projected/5d07caad-64bc-46d3-9b3d-96a7c3c69e24-kube-api-access-9s2l4\") pod \"model-serving-api-54d89bd5fc-cx8rc\" (UID: \"5d07caad-64bc-46d3-9b3d-96a7c3c69e24\") " pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:18.389973 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:18.389938    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k" event={"ID":"c905ec0d-2626-4e38-918d-df403ab57f76","Type":"ContainerStarted","Data":"3774d7367649c65999d903ce0663a1ed21c03ec9fa7fee248cd90dc3b34d671e"}
Sep 11 14:06:18.601458 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:18.601424    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d07caad-64bc-46d3-9b3d-96a7c3c69e24-tls-certs\") pod \"model-serving-api-54d89bd5fc-cx8rc\" (UID: \"5d07caad-64bc-46d3-9b3d-96a7c3c69e24\") " pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:18.603860 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:18.603839    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5d07caad-64bc-46d3-9b3d-96a7c3c69e24-tls-certs\") pod \"model-serving-api-54d89bd5fc-cx8rc\" (UID: \"5d07caad-64bc-46d3-9b3d-96a7c3c69e24\") " pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:18.869306 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:18.869270    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:19.155365 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:19.155270    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-cx8rc"]
Sep 11 14:06:19.158103 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:06:19.158054    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d07caad_64bc_46d3_9b3d_96a7c3c69e24.slice/crio-171dcea59a18fc0365687a7d28d345aa03f209e936976c4973735cde58b60b28 WatchSource:0}: Error finding container 171dcea59a18fc0365687a7d28d345aa03f209e936976c4973735cde58b60b28: Status 404 returned error can't find the container with id 171dcea59a18fc0365687a7d28d345aa03f209e936976c4973735cde58b60b28
Sep 11 14:06:19.395680 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:19.395612    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc" event={"ID":"5d07caad-64bc-46d3-9b3d-96a7c3c69e24","Type":"ContainerStarted","Data":"171dcea59a18fc0365687a7d28d345aa03f209e936976c4973735cde58b60b28"}
Sep 11 14:06:21.410040 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:21.409958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k" event={"ID":"c905ec0d-2626-4e38-918d-df403ab57f76","Type":"ContainerStarted","Data":"97155f68b33123b2dcc6cce0dfb2db7b4fec65d76b62a0842b3cbea66189ab62"}
Sep 11 14:06:21.410522 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:21.410338    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:23.426767 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:23.426730    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc" event={"ID":"5d07caad-64bc-46d3-9b3d-96a7c3c69e24","Type":"ContainerStarted","Data":"7fc3c4ac95f52fa5bb77b7a8ceac55620c8f498a8593123ea57f07ea173cd2ee"}
Sep 11 14:06:23.427107 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:23.426820    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:06:23.447892 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:23.447847    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k" podStartSLOduration=3.746628443 podStartE2EDuration="6.447830144s" totalImagesPullingTime="2.701201701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:21.429289448 +0000 UTC m=+531.766865009" watchObservedRunningTime="2026-09-11 14:06:23.447830144 +0000 UTC m=+533.785405706"
Sep 11 14:06:23.448061 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:23.447945    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc" podStartSLOduration=3.03095823 podStartE2EDuration="6.447937526s" totalImagesPullingTime="3.416979296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:23.445760536 +0000 UTC m=+533.783336095" watchObservedRunningTime="2026-09-11 14:06:23.447937526 +0000 UTC m=+533.785513089"
Sep 11 14:06:24.968043 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.968009    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"]
Sep 11 14:06:24.972935 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.972919    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:24.977370 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.977349    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 14:06:24.978249 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.978230    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 14:06:24.978345 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.978275    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 14:06:24.978345 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.978278    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fbfbr\""
Sep 11 14:06:24.978547 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.978532    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 14:06:24.978746 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.978728    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 14:06:24.978861 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.978768    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 14:06:24.994757 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:24.994733    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"]
Sep 11 14:06:25.056598 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.056567    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.056598 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.056599    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.056854 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.056647    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.056854 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.056762    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9zmp\" (UniqueName: \"kubernetes.io/projected/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-kube-api-access-z9zmp\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.056854 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.056817    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.056854 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.056844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.056989 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.056904    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.157611 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.157581    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.157611 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.157610    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.157852 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.157653    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.157852 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.157690    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9zmp\" (UniqueName: \"kubernetes.io/projected/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-kube-api-access-z9zmp\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.157852 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.157715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.157852 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.157731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.157852 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.157763    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.158681 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.158650    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.160108 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.160079    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.160214 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.160111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.160214 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.160118    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.160214 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.160155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.170951 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.170929    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.172943 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.172926    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9zmp\" (UniqueName: \"kubernetes.io/projected/d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7-kube-api-access-z9zmp\") pod \"istiod-openshift-gateway-5677ccb49b-7x9t7\" (UID: \"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.282836 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.282769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:25.583915 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:25.583890    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"]
Sep 11 14:06:25.585855 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:06:25.585830    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd186a0c3_98ab_4c8a_be6d_7dc1d0bceee7.slice/crio-87e047a12523dfe3c92800bdfb8ef2cf61f7e90a40c97da2661e4d04e2d4bade WatchSource:0}: Error finding container 87e047a12523dfe3c92800bdfb8ef2cf61f7e90a40c97da2661e4d04e2d4bade: Status 404 returned error can't find the container with id 87e047a12523dfe3c92800bdfb8ef2cf61f7e90a40c97da2661e4d04e2d4bade
Sep 11 14:06:26.441909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:26.441868    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7" event={"ID":"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7","Type":"ContainerStarted","Data":"87e047a12523dfe3c92800bdfb8ef2cf61f7e90a40c97da2661e4d04e2d4bade"}
Sep 11 14:06:28.403287 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:28.403248    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 14:06:28.403686 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:28.403332    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 11 14:06:29.461715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:29.461675    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7" event={"ID":"d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7","Type":"ContainerStarted","Data":"c89fe1482427e5bc8fc6f5ab3ba39a57277ae33e5f5ca525487cb88e7f55c85d"}
Sep 11 14:06:29.462148 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:29.461827    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:29.463377 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:29.463351    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-7x9t7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 14:06:29.463489 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:29.463394    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7" podUID="d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 14:06:29.523171 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:29.523122    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7" podStartSLOduration=2.707995332 podStartE2EDuration="5.523107749s" totalImagesPullingTime="2.815112417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:06:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:06:29.521615604 +0000 UTC m=+539.859191166" watchObservedRunningTime="2026-09-11 14:06:29.523107749 +0000 UTC m=+539.860683309"
Sep 11 14:06:30.467081 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:30.467055    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-7x9t7"
Sep 11 14:06:32.417236 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:32.417208    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kzl4k"
Sep 11 14:06:34.434701 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:06:34.434672    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-cx8rc"
Sep 11 14:09:09.714703 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.714665    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f955487ff-gzphm"]
Sep 11 14:09:09.717827 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.717811    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.720726 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.720701    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dkzmk\""
Sep 11 14:09:09.721366 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.721342    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 14:09:09.721501 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.721391    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 14:09:09.723322 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.723296    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 14:09:09.734518 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.734496    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f955487ff-gzphm"]
Sep 11 14:09:09.820553 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.820515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kxjf\" (UniqueName: \"kubernetes.io/projected/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-kube-api-access-9kxjf\") pod \"maas-controller-f955487ff-gzphm\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") " pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.820737 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.820562    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-metrics-certs\") pod \"maas-controller-f955487ff-gzphm\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") " pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.820737 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.820584    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-webhook-cert\") pod \"maas-controller-f955487ff-gzphm\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") " pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.921103 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.921069    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-rxzgw"]
Sep 11 14:09:09.921412 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.921391    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-metrics-certs\") pod \"maas-controller-f955487ff-gzphm\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") " pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.921499 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.921434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-webhook-cert\") pod \"maas-controller-f955487ff-gzphm\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") " pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.921567 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.921550    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kxjf\" (UniqueName: \"kubernetes.io/projected/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-kube-api-access-9kxjf\") pod \"maas-controller-f955487ff-gzphm\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") " pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.924109 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.924084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-webhook-cert\") pod \"maas-controller-f955487ff-gzphm\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") " pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.924109 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.924108    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-metrics-certs\") pod \"maas-controller-f955487ff-gzphm\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") " pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.924797 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.924779    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:09.929261 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.929245    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kxjf\" (UniqueName: \"kubernetes.io/projected/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-kube-api-access-9kxjf\") pod \"maas-controller-f955487ff-gzphm\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") " pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:09.936891 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:09.936869    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-rxzgw"]
Sep 11 14:09:10.022578 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.022506    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-webhook-cert\") pod \"maas-controller-58c98bd7bd-rxzgw\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") " pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.022733 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.022581    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9nx\" (UniqueName: \"kubernetes.io/projected/fb99cd57-e8b3-493b-af8c-a61e6b450175-kube-api-access-gx9nx\") pod \"maas-controller-58c98bd7bd-rxzgw\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") " pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.022733 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.022615    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-metrics-certs\") pod \"maas-controller-58c98bd7bd-rxzgw\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") " pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.028464 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.028437    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:10.123323 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.123288    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9nx\" (UniqueName: \"kubernetes.io/projected/fb99cd57-e8b3-493b-af8c-a61e6b450175-kube-api-access-gx9nx\") pod \"maas-controller-58c98bd7bd-rxzgw\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") " pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.123463 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.123335    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-metrics-certs\") pod \"maas-controller-58c98bd7bd-rxzgw\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") " pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.123463 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.123370    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-webhook-cert\") pod \"maas-controller-58c98bd7bd-rxzgw\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") " pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.125698 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.125673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-webhook-cert\") pod \"maas-controller-58c98bd7bd-rxzgw\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") " pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.125806 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.125721    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-metrics-certs\") pod \"maas-controller-58c98bd7bd-rxzgw\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") " pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.134335 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.134317    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9nx\" (UniqueName: \"kubernetes.io/projected/fb99cd57-e8b3-493b-af8c-a61e6b450175-kube-api-access-gx9nx\") pod \"maas-controller-58c98bd7bd-rxzgw\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") " pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.253274 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.253245    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:10.301947 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.301914    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f955487ff-gzphm"]
Sep 11 14:09:10.305609 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:09:10.305344    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf801ccc_5088_4faa_b1e9_27a776ecb7d1.slice/crio-d65a0b92146b4cd386b91852d7700f27d3d1bff22e05288c786947019df41180 WatchSource:0}: Error finding container d65a0b92146b4cd386b91852d7700f27d3d1bff22e05288c786947019df41180: Status 404 returned error can't find the container with id d65a0b92146b4cd386b91852d7700f27d3d1bff22e05288c786947019df41180
Sep 11 14:09:10.306748 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.306706    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:09:10.529901 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:10.529736    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-rxzgw"]
Sep 11 14:09:10.531903 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:09:10.531879    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb99cd57_e8b3_493b_af8c_a61e6b450175.slice/crio-ba3c09e41c9fe6e25a67c59b6776eb898d7553740e5b5a7e6e0947f733daa870 WatchSource:0}: Error finding container ba3c09e41c9fe6e25a67c59b6776eb898d7553740e5b5a7e6e0947f733daa870: Status 404 returned error can't find the container with id ba3c09e41c9fe6e25a67c59b6776eb898d7553740e5b5a7e6e0947f733daa870
Sep 11 14:09:11.166571 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:11.166481    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw" event={"ID":"fb99cd57-e8b3-493b-af8c-a61e6b450175","Type":"ContainerStarted","Data":"ba3c09e41c9fe6e25a67c59b6776eb898d7553740e5b5a7e6e0947f733daa870"}
Sep 11 14:09:11.169173 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:11.169113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f955487ff-gzphm" event={"ID":"bf801ccc-5088-4faa-b1e9-27a776ecb7d1","Type":"ContainerStarted","Data":"d65a0b92146b4cd386b91852d7700f27d3d1bff22e05288c786947019df41180"}
Sep 11 14:09:13.182481 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:13.182440    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw" event={"ID":"fb99cd57-e8b3-493b-af8c-a61e6b450175","Type":"ContainerStarted","Data":"489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959"}
Sep 11 14:09:13.182995 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:13.182729    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:13.184141 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:13.184110    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f955487ff-gzphm" event={"ID":"bf801ccc-5088-4faa-b1e9-27a776ecb7d1","Type":"ContainerStarted","Data":"f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0"}
Sep 11 14:09:13.184276 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:13.184198    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:13.216239 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:13.216186    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw" podStartSLOduration=1.72406727 podStartE2EDuration="4.216171806s" totalImagesPullingTime="2.492104536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:09:13.213895861 +0000 UTC m=+703.551471434" watchObservedRunningTime="2026-09-11 14:09:13.216171806 +0000 UTC m=+703.553747366"
Sep 11 14:09:13.252838 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:13.252791    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f955487ff-gzphm" podStartSLOduration=1.540697872 podStartE2EDuration="4.252777985s" totalImagesPullingTime="2.712080113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:09:13.249531227 +0000 UTC m=+703.587106787" watchObservedRunningTime="2026-09-11 14:09:13.252777985 +0000 UTC m=+703.590353549"
Sep 11 14:09:24.190410 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.190328    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:09:24.190410 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.190392    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:24.256883 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.256844    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f955487ff-gzphm"]
Sep 11 14:09:24.257180 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.257097    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f955487ff-gzphm" podUID="bf801ccc-5088-4faa-b1e9-27a776ecb7d1" containerName="manager" containerID="cri-o://f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0" gracePeriod=30
Sep 11 14:09:24.499199 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.499172    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:24.654293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.654261    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-metrics-certs\") pod \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") "
Sep 11 14:09:24.654447 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.654330    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-webhook-cert\") pod \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") "
Sep 11 14:09:24.654447 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.654370    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kxjf\" (UniqueName: \"kubernetes.io/projected/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-kube-api-access-9kxjf\") pod \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\" (UID: \"bf801ccc-5088-4faa-b1e9-27a776ecb7d1\") "
Sep 11 14:09:24.656407 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.656375    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bf801ccc-5088-4faa-b1e9-27a776ecb7d1" (UID: "bf801ccc-5088-4faa-b1e9-27a776ecb7d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:09:24.656513 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.656417    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-kube-api-access-9kxjf" (OuterVolumeSpecName: "kube-api-access-9kxjf") pod "bf801ccc-5088-4faa-b1e9-27a776ecb7d1" (UID: "bf801ccc-5088-4faa-b1e9-27a776ecb7d1"). InnerVolumeSpecName "kube-api-access-9kxjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:09:24.656513 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.656474    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bf801ccc-5088-4faa-b1e9-27a776ecb7d1" (UID: "bf801ccc-5088-4faa-b1e9-27a776ecb7d1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:09:24.755597 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.755528    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:09:24.755597 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.755553    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-webhook-cert\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:09:24.755597 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:24.755563    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kxjf\" (UniqueName: \"kubernetes.io/projected/bf801ccc-5088-4faa-b1e9-27a776ecb7d1-kube-api-access-9kxjf\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:09:25.243742 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:25.243707    2564 generic.go:356] "Generic (PLEG): container finished" podID="bf801ccc-5088-4faa-b1e9-27a776ecb7d1" containerID="f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0" exitCode=0
Sep 11 14:09:25.244113 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:25.243754    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f955487ff-gzphm"
Sep 11 14:09:25.244113 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:25.243764    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f955487ff-gzphm" event={"ID":"bf801ccc-5088-4faa-b1e9-27a776ecb7d1","Type":"ContainerDied","Data":"f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0"}
Sep 11 14:09:25.244113 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:25.243813    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f955487ff-gzphm" event={"ID":"bf801ccc-5088-4faa-b1e9-27a776ecb7d1","Type":"ContainerDied","Data":"d65a0b92146b4cd386b91852d7700f27d3d1bff22e05288c786947019df41180"}
Sep 11 14:09:25.244113 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:25.243843    2564 scope.go:117] "RemoveContainer" containerID="f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0"
Sep 11 14:09:25.253135 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:25.253116    2564 scope.go:117] "RemoveContainer" containerID="f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0"
Sep 11 14:09:25.253401 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:09:25.253383    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0\": container with ID starting with f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0 not found: ID does not exist" containerID="f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0"
Sep 11 14:09:25.253452 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:25.253410    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0"} err="failed to get container status \"f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0\": rpc error: code = NotFound desc = could not find container \"f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0\": container with ID starting with f660e0df5ad84fd3380db4d88488101a5e16864280f5229081cdc679b40a62f0 not found: ID does not exist"
Sep 11 14:09:25.268965 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:25.268940    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f955487ff-gzphm"]
Sep 11 14:09:25.276622 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:25.276599    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f955487ff-gzphm"]
Sep 11 14:09:26.274858 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:26.274826    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf801ccc-5088-4faa-b1e9-27a776ecb7d1" path="/var/lib/kubelet/pods/bf801ccc-5088-4faa-b1e9-27a776ecb7d1/volumes"
Sep 11 14:09:45.838258 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.838218    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"]
Sep 11 14:09:45.838738 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.838721    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf801ccc-5088-4faa-b1e9-27a776ecb7d1" containerName="manager"
Sep 11 14:09:45.838738 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.838739    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf801ccc-5088-4faa-b1e9-27a776ecb7d1" containerName="manager"
Sep 11 14:09:45.838829 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.838809    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf801ccc-5088-4faa-b1e9-27a776ecb7d1" containerName="manager"
Sep 11 14:09:45.843317 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.843294    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:45.884076 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.884049    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 14:09:45.884076 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.884070    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 14:09:45.884302 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.884289    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 14:09:45.884509 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.884496    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-ktm9c\""
Sep 11 14:09:45.903972 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.903946    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 14:09:45.915686 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.915666    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"]
Sep 11 14:09:45.939727 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.939703    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-maas-api-tls\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:45.939830 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.939736    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-metrics-certs\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:45.939830 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.939763    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89xmr\" (UniqueName: \"kubernetes.io/projected/f89b66b3-7024-4432-884a-e5ddd4cf26bb-kube-api-access-89xmr\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:45.939935 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:45.939815    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f89b66b3-7024-4432-884a-e5ddd4cf26bb-openshift-service-ca\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.040441 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.040412    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-maas-api-tls\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.040617 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.040449    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-metrics-certs\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.040617 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.040468    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89xmr\" (UniqueName: \"kubernetes.io/projected/f89b66b3-7024-4432-884a-e5ddd4cf26bb-kube-api-access-89xmr\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.040617 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.040514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f89b66b3-7024-4432-884a-e5ddd4cf26bb-openshift-service-ca\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.041243 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.041221    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f89b66b3-7024-4432-884a-e5ddd4cf26bb-openshift-service-ca\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.042985 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.042964    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-metrics-certs\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.043076 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.042993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-maas-api-tls\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.062551 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.062526    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89xmr\" (UniqueName: \"kubernetes.io/projected/f89b66b3-7024-4432-884a-e5ddd4cf26bb-kube-api-access-89xmr\") pod \"maas-api-d8d795dc5-z6bxh\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") " pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.154190 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.154109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:46.452000 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:46.451976    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"]
Sep 11 14:09:46.454392 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:09:46.454363    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf89b66b3_7024_4432_884a_e5ddd4cf26bb.slice/crio-ab16f8b6d4f0ccdf5e2c68e937790f35754883ceffdc48c9aeb533a93c16f4ad WatchSource:0}: Error finding container ab16f8b6d4f0ccdf5e2c68e937790f35754883ceffdc48c9aeb533a93c16f4ad: Status 404 returned error can't find the container with id ab16f8b6d4f0ccdf5e2c68e937790f35754883ceffdc48c9aeb533a93c16f4ad
Sep 11 14:09:47.062310 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.062280    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"]
Sep 11 14:09:47.067790 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.067770    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:09:47.077499 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.077477    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 14:09:47.107290 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.107267    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"]
Sep 11 14:09:47.149415 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.149375    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70ee0d68-d5ee-4abe-8b6b-80142836b2aa-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-4jzpw\" (UID: \"70ee0d68-d5ee-4abe-8b6b-80142836b2aa\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:09:47.149557 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.149427    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt5bw\" (UniqueName: \"kubernetes.io/projected/70ee0d68-d5ee-4abe-8b6b-80142836b2aa-kube-api-access-gt5bw\") pod \"payload-pre-processing-59fbc47bf8-4jzpw\" (UID: \"70ee0d68-d5ee-4abe-8b6b-80142836b2aa\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:09:47.250030 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.249977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70ee0d68-d5ee-4abe-8b6b-80142836b2aa-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-4jzpw\" (UID: \"70ee0d68-d5ee-4abe-8b6b-80142836b2aa\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:09:47.250211 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.250052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt5bw\" (UniqueName: \"kubernetes.io/projected/70ee0d68-d5ee-4abe-8b6b-80142836b2aa-kube-api-access-gt5bw\") pod \"payload-pre-processing-59fbc47bf8-4jzpw\" (UID: \"70ee0d68-d5ee-4abe-8b6b-80142836b2aa\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:09:47.250907 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.250887    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/70ee0d68-d5ee-4abe-8b6b-80142836b2aa-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-4jzpw\" (UID: \"70ee0d68-d5ee-4abe-8b6b-80142836b2aa\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:09:47.275060 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.275034    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt5bw\" (UniqueName: \"kubernetes.io/projected/70ee0d68-d5ee-4abe-8b6b-80142836b2aa-kube-api-access-gt5bw\") pod \"payload-pre-processing-59fbc47bf8-4jzpw\" (UID: \"70ee0d68-d5ee-4abe-8b6b-80142836b2aa\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:09:47.341489 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.341413    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh" event={"ID":"f89b66b3-7024-4432-884a-e5ddd4cf26bb","Type":"ContainerStarted","Data":"ab16f8b6d4f0ccdf5e2c68e937790f35754883ceffdc48c9aeb533a93c16f4ad"}
Sep 11 14:09:47.377493 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.377452    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:09:47.671432 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:47.671262    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"]
Sep 11 14:09:48.061942 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:09:48.061915    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ee0d68_d5ee_4abe_8b6b_80142836b2aa.slice/crio-7e3a87cfbc59c0913a660645b926dc3e868e7da9afa8cdd141f9f6d55b2400eb WatchSource:0}: Error finding container 7e3a87cfbc59c0913a660645b926dc3e868e7da9afa8cdd141f9f6d55b2400eb: Status 404 returned error can't find the container with id 7e3a87cfbc59c0913a660645b926dc3e868e7da9afa8cdd141f9f6d55b2400eb
Sep 11 14:09:48.347095 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:48.347004    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh" event={"ID":"f89b66b3-7024-4432-884a-e5ddd4cf26bb","Type":"ContainerStarted","Data":"f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718"}
Sep 11 14:09:48.347493 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:48.347113    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:48.348368 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:48.348345    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw" event={"ID":"70ee0d68-d5ee-4abe-8b6b-80142836b2aa","Type":"ContainerStarted","Data":"7e3a87cfbc59c0913a660645b926dc3e868e7da9afa8cdd141f9f6d55b2400eb"}
Sep 11 14:09:48.372596 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:48.372539    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh" podStartSLOduration=1.719846245 podStartE2EDuration="3.372526324s" totalImagesPullingTime="1.652680079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:09:48.370738848 +0000 UTC m=+738.708314407" watchObservedRunningTime="2026-09-11 14:09:48.372526324 +0000 UTC m=+738.710101880"
Sep 11 14:09:51.364313 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:51.364275    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw" event={"ID":"70ee0d68-d5ee-4abe-8b6b-80142836b2aa","Type":"ContainerStarted","Data":"2aaf98b430b5b323e98fb744ee549bf56eaefa949056e7b6bf01c733c8c07b4f"}
Sep 11 14:09:51.364772 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:51.364366    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:09:54.359597 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:54.359568    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:09:54.385931 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:09:54.385881    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw" podStartSLOduration=5.136097051 podStartE2EDuration="7.385865661s" totalImagesPullingTime="2.24976861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:09:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:09:51.392807347 +0000 UTC m=+741.730382911" watchObservedRunningTime="2026-09-11 14:09:54.385865661 +0000 UTC m=+744.723441241"
Sep 11 14:10:02.372130 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.372095    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-4jzpw"
Sep 11 14:10:02.613988 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.613955    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"]
Sep 11 14:10:02.617347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.617328    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.624443 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.624419    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 14:10:02.624586 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.624421    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7lnrv\""
Sep 11 14:10:02.624713 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.624694    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 14:10:02.625176 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.625161    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 11 14:10:02.640753 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.640729    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"]
Sep 11 14:10:02.681170 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.681111    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frzz7\" (UniqueName: \"kubernetes.io/projected/be65d086-8894-407a-bad9-6421ec87abeb-kube-api-access-frzz7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.681170 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.681155    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.681353 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.681179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.681353 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.681249    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be65d086-8894-407a-bad9-6421ec87abeb-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.681353 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.681293    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.681353 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.681320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.782489 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.782443    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frzz7\" (UniqueName: \"kubernetes.io/projected/be65d086-8894-407a-bad9-6421ec87abeb-kube-api-access-frzz7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.782665 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.782527    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.782665 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.782553    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.782665 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.782585    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be65d086-8894-407a-bad9-6421ec87abeb-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.782665 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.782605    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.782665 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.782655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.783019 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.782996    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.783082 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.783011    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.783082 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.783064    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.784932 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.784912    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be65d086-8894-407a-bad9-6421ec87abeb-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.785114 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.785096    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be65d086-8894-407a-bad9-6421ec87abeb-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.791442 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.791421    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frzz7\" (UniqueName: \"kubernetes.io/projected/be65d086-8894-407a-bad9-6421ec87abeb-kube-api-access-frzz7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz\" (UID: \"be65d086-8894-407a-bad9-6421ec87abeb\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:02.927497 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:02.927419    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:03.207652 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:03.207613    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"]
Sep 11 14:10:03.210043 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:10:03.210020    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe65d086_8894_407a_bad9_6421ec87abeb.slice/crio-e010b291d22d401829c166156312bafb2dd4eeb9ae54e5549de2b5804d4feab7 WatchSource:0}: Error finding container e010b291d22d401829c166156312bafb2dd4eeb9ae54e5549de2b5804d4feab7: Status 404 returned error can't find the container with id e010b291d22d401829c166156312bafb2dd4eeb9ae54e5549de2b5804d4feab7
Sep 11 14:10:03.418574 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:03.418544    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz" event={"ID":"be65d086-8894-407a-bad9-6421ec87abeb","Type":"ContainerStarted","Data":"e010b291d22d401829c166156312bafb2dd4eeb9ae54e5549de2b5804d4feab7"}
Sep 11 14:10:04.424074 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:04.424044    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz" event={"ID":"be65d086-8894-407a-bad9-6421ec87abeb","Type":"ContainerStarted","Data":"083c332b4fc5b27d6604054c81841b9e466329383dbfc31b7927731950807209"}
Sep 11 14:10:04.454534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:04.454404    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz" podStartSLOduration=1.342815421 podStartE2EDuration="2.454387662s" totalImagesPullingTime="1.111572241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:10:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:10:04.452597205 +0000 UTC m=+754.790172766" watchObservedRunningTime="2026-09-11 14:10:04.454387662 +0000 UTC m=+754.791963247"
Sep 11 14:10:12.928195 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:12.928161    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:12.928571 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:12.928247    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:12.940535 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:12.940514    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:13.474936 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:13.474910    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz"
Sep 11 14:10:52.299177 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.299145    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"]
Sep 11 14:10:52.303558 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.303542    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.309506 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.309481    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 14:10:52.337497 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.337474    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"]
Sep 11 14:10:52.416968 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.416930    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m99d6\" (UniqueName: \"kubernetes.io/projected/b1913686-57ef-458e-b84a-990becea2844-kube-api-access-m99d6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.417137 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.416986    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.417137 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.417049    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.417137 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.417116    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1913686-57ef-458e-b84a-990becea2844-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.417270 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.417140    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.417270 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.417246    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.518039 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.518005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1913686-57ef-458e-b84a-990becea2844-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.518039 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.518045    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.518342 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.518159    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.518342 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.518186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m99d6\" (UniqueName: \"kubernetes.io/projected/b1913686-57ef-458e-b84a-990becea2844-kube-api-access-m99d6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.518342 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.518215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.518342 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.518233    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.518586 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.518556    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.518669 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.518612    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.518669 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.518660    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.520408 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.520381    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b1913686-57ef-458e-b84a-990becea2844-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.520707 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.520687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b1913686-57ef-458e-b84a-990becea2844-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.532363 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.532338    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m99d6\" (UniqueName: \"kubernetes.io/projected/b1913686-57ef-458e-b84a-990becea2844-kube-api-access-m99d6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5\" (UID: \"b1913686-57ef-458e-b84a-990becea2844\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.613529 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.613462    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:10:52.892748 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:52.892722    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"]
Sep 11 14:10:52.895226 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:10:52.895203    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1913686_57ef_458e_b84a_990becea2844.slice/crio-f85644f644b35265ae650cbbf7587201be94fb85b23c6cc3ce4798da5c27b43b WatchSource:0}: Error finding container f85644f644b35265ae650cbbf7587201be94fb85b23c6cc3ce4798da5c27b43b: Status 404 returned error can't find the container with id f85644f644b35265ae650cbbf7587201be94fb85b23c6cc3ce4798da5c27b43b
Sep 11 14:10:53.642584 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:53.642549    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5" event={"ID":"b1913686-57ef-458e-b84a-990becea2844","Type":"ContainerStarted","Data":"dfd8d14c3e9da3379a6bc31225120884beddbf674fd982730a3f3d9bf77a98ab"}
Sep 11 14:10:53.643022 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:10:53.642601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5" event={"ID":"b1913686-57ef-458e-b84a-990becea2844","Type":"ContainerStarted","Data":"f85644f644b35265ae650cbbf7587201be94fb85b23c6cc3ce4798da5c27b43b"}
Sep 11 14:11:02.614166 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:11:02.614129    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:11:02.614569 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:11:02.614188    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:11:02.626752 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:11:02.626726    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:11:02.656244 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:11:02.656200    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5" podStartSLOduration=10.474304217 podStartE2EDuration="10.656181818s" totalImagesPullingTime="181.877601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:10:53.675006734 +0000 UTC m=+804.012582294" watchObservedRunningTime="2026-09-11 14:11:02.656181818 +0000 UTC m=+812.993757379"
Sep 11 14:11:02.696011 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:11:02.695980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5"
Sep 11 14:13:18.356338 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.356301    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"]
Sep 11 14:13:18.360150 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.360128    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.366719 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.366698    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-be0c80-serving-cert\""
Sep 11 14:13:18.387966 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.387939    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"]
Sep 11 14:13:18.475646 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.475593    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sngk7\" (UniqueName: \"kubernetes.io/projected/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-kube-api-access-sngk7\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.475794 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.475724    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.475794 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.475755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-metrics-certs\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.475794 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.475790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-maas-api-tls\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.576686 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.576656    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sngk7\" (UniqueName: \"kubernetes.io/projected/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-kube-api-access-sngk7\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.576848 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.576724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.576848 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.576746    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-metrics-certs\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.576848 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.576775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-maas-api-tls\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.576974 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:13:18.576892    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-serving-cert: secret "maas-api-e2e-inf-a-be0c80-serving-cert" not found
Sep 11 14:13:18.576974 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:13:18.576941    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-maas-api-tls podName:204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0 nodeName:}" failed. No retries permitted until 2026-09-11 14:13:19.076926694 +0000 UTC m=+949.414502235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-maas-api-tls") pod "maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb" (UID: "204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0") : secret "maas-api-e2e-inf-a-be0c80-serving-cert" not found
Sep 11 14:13:18.577405 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.577381    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.579212 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.579188    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-metrics-certs\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:18.594408 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:18.594376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sngk7\" (UniqueName: \"kubernetes.io/projected/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-kube-api-access-sngk7\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:19.081775 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.081743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-maas-api-tls\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:19.084161 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.084135    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-maas-api-tls\") pod \"maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:19.273049 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.273007    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:19.366467 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.366394    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"]
Sep 11 14:13:19.370209 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.370191    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:19.372679 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.372623    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-be0c80-dockercfg-l752g\""
Sep 11 14:13:19.375340 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.375322    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-be0c80\""
Sep 11 14:13:19.394634 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.394605    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"]
Sep 11 14:13:19.459012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.458980    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"]
Sep 11 14:13:19.462261 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.462241    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:19.485907 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.485878    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsk4x\" (UniqueName: \"kubernetes.io/projected/6108161b-a6ee-4ba0-a061-277e90658ea2-kube-api-access-bsk4x\") pod \"payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk\" (UID: \"6108161b-a6ee-4ba0-a061-277e90658ea2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:19.486092 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.486011    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6108161b-a6ee-4ba0-a061-277e90658ea2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk\" (UID: \"6108161b-a6ee-4ba0-a061-277e90658ea2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:19.488407 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.488382    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"]
Sep 11 14:13:19.559060 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.559034    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"]
Sep 11 14:13:19.561616 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:13:19.561577    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod204c6ce5_5526_4ffc_a3b2_aee8c5a6d2b0.slice/crio-8c7678f7f64efec2912e8787f15f00ae200e1c1a4d85daa23ce8756f14f2a4fe WatchSource:0}: Error finding container 8c7678f7f64efec2912e8787f15f00ae200e1c1a4d85daa23ce8756f14f2a4fe: Status 404 returned error can't find the container with id 8c7678f7f64efec2912e8787f15f00ae200e1c1a4d85daa23ce8756f14f2a4fe
Sep 11 14:13:19.586741 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.586714    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsk4x\" (UniqueName: \"kubernetes.io/projected/6108161b-a6ee-4ba0-a061-277e90658ea2-kube-api-access-bsk4x\") pod \"payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk\" (UID: \"6108161b-a6ee-4ba0-a061-277e90658ea2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:19.586837 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.586789    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e449eacc-b976-40b7-9c53-d5bc817f18d2-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4\" (UID: \"e449eacc-b976-40b7-9c53-d5bc817f18d2\") " pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:19.586837 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.586827    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6108161b-a6ee-4ba0-a061-277e90658ea2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk\" (UID: \"6108161b-a6ee-4ba0-a061-277e90658ea2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:19.586955 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.586913    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r9cd\" (UniqueName: \"kubernetes.io/projected/e449eacc-b976-40b7-9c53-d5bc817f18d2-kube-api-access-9r9cd\") pod \"payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4\" (UID: \"e449eacc-b976-40b7-9c53-d5bc817f18d2\") " pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:19.587562 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.587545    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6108161b-a6ee-4ba0-a061-277e90658ea2-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk\" (UID: \"6108161b-a6ee-4ba0-a061-277e90658ea2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:19.599113 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.599083    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsk4x\" (UniqueName: \"kubernetes.io/projected/6108161b-a6ee-4ba0-a061-277e90658ea2-kube-api-access-bsk4x\") pod \"payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk\" (UID: \"6108161b-a6ee-4ba0-a061-277e90658ea2\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:19.680350 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.680271    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:19.687473 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.687450    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e449eacc-b976-40b7-9c53-d5bc817f18d2-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4\" (UID: \"e449eacc-b976-40b7-9c53-d5bc817f18d2\") " pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:19.687576 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.687514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r9cd\" (UniqueName: \"kubernetes.io/projected/e449eacc-b976-40b7-9c53-d5bc817f18d2-kube-api-access-9r9cd\") pod \"payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4\" (UID: \"e449eacc-b976-40b7-9c53-d5bc817f18d2\") " pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:19.688220 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.688200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e449eacc-b976-40b7-9c53-d5bc817f18d2-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4\" (UID: \"e449eacc-b976-40b7-9c53-d5bc817f18d2\") " pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:19.697347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.697322    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r9cd\" (UniqueName: \"kubernetes.io/projected/e449eacc-b976-40b7-9c53-d5bc817f18d2-kube-api-access-9r9cd\") pod \"payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4\" (UID: \"e449eacc-b976-40b7-9c53-d5bc817f18d2\") " pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:19.772067 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.772039    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:19.960446 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:19.960418    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"]
Sep 11 14:13:20.060586 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.060563    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"]
Sep 11 14:13:20.062484 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:13:20.062453    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode449eacc_b976_40b7_9c53_d5bc817f18d2.slice/crio-63370817319ff891326a144e6f297d0253858468e145ab86b4af63d1a8c9a489 WatchSource:0}: Error finding container 63370817319ff891326a144e6f297d0253858468e145ab86b4af63d1a8c9a489: Status 404 returned error can't find the container with id 63370817319ff891326a144e6f297d0253858468e145ab86b4af63d1a8c9a489
Sep 11 14:13:20.322299 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.322262    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk" event={"ID":"6108161b-a6ee-4ba0-a061-277e90658ea2","Type":"ContainerStarted","Data":"dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e"}
Sep 11 14:13:20.322488 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.322330    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:20.322488 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.322355    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk" event={"ID":"6108161b-a6ee-4ba0-a061-277e90658ea2","Type":"ContainerStarted","Data":"ca0c5328f3e59638c2dfaca89becf9d88e10d85c4f8d92e2162df833c335a172"}
Sep 11 14:13:20.324047 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.324016    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4" event={"ID":"e449eacc-b976-40b7-9c53-d5bc817f18d2","Type":"ContainerStarted","Data":"ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f"}
Sep 11 14:13:20.324195 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.324069    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4" event={"ID":"e449eacc-b976-40b7-9c53-d5bc817f18d2","Type":"ContainerStarted","Data":"63370817319ff891326a144e6f297d0253858468e145ab86b4af63d1a8c9a489"}
Sep 11 14:13:20.324195 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.324098    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:20.325751 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.325726    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb" event={"ID":"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0","Type":"ContainerStarted","Data":"83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e"}
Sep 11 14:13:20.325939 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.325772    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb" event={"ID":"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0","Type":"ContainerStarted","Data":"8c7678f7f64efec2912e8787f15f00ae200e1c1a4d85daa23ce8756f14f2a4fe"}
Sep 11 14:13:20.325939 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.325802    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:20.393291 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.393230    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb" podStartSLOduration=1.9829966749999999 podStartE2EDuration="2.393209094s" totalImagesPullingTime="410.212419ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:20.392849388 +0000 UTC m=+950.730424946" watchObservedRunningTime="2026-09-11 14:13:20.393209094 +0000 UTC m=+950.730784659"
Sep 11 14:13:20.393737 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.393347    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk" podStartSLOduration=1.393340064 podStartE2EDuration="1.393340064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:20.352160614 +0000 UTC m=+950.689736176" watchObservedRunningTime="2026-09-11 14:13:20.393340064 +0000 UTC m=+950.730915628"
Sep 11 14:13:20.434354 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:20.434303    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4" podStartSLOduration=1.434286562 podStartE2EDuration="1.434286562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:20.434036711 +0000 UTC m=+950.771612271" watchObservedRunningTime="2026-09-11 14:13:20.434286562 +0000 UTC m=+950.771862121"
Sep 11 14:13:26.337029 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:26.337000    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:13:27.251408 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.251372    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"]
Sep 11 14:13:27.254602 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.254582    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.257889 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.257852    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-1dc719b7-serving-cert\""
Sep 11 14:13:27.276818 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.276784    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"]
Sep 11 14:13:27.359008 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.358977    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-openshift-service-ca\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.359387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.359027    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-metrics-certs\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.359387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.359053    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nf7p\" (UniqueName: \"kubernetes.io/projected/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-kube-api-access-8nf7p\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.359387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.359196    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-maas-api-tls\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.460582 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.460551    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-openshift-service-ca\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.460752 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.460598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-metrics-certs\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.460752 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.460619    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nf7p\" (UniqueName: \"kubernetes.io/projected/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-kube-api-access-8nf7p\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.460752 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.460689    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-maas-api-tls\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.460928 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:13:27.460787    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-serving-cert: secret "maas-api-e2e-ait-1dc719b7-serving-cert" not found
Sep 11 14:13:27.460928 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:13:27.460847    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-maas-api-tls podName:fbee1e10-fab3-4fa5-ac08-cca21ab16d61 nodeName:}" failed. No retries permitted until 2026-09-11 14:13:27.960829687 +0000 UTC m=+958.298405246 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-maas-api-tls") pod "maas-api-e2e-ait-1dc719b7-589b769fb-244xm" (UID: "fbee1e10-fab3-4fa5-ac08-cca21ab16d61") : secret "maas-api-e2e-ait-1dc719b7-serving-cert" not found
Sep 11 14:13:27.461400 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.461376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-openshift-service-ca\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.463028 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.463008    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-metrics-certs\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.475578 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.475553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nf7p\" (UniqueName: \"kubernetes.io/projected/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-kube-api-access-8nf7p\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.966827 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.966793    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-maas-api-tls\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:27.969204 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:27.969176    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-maas-api-tls\") pod \"maas-api-e2e-ait-1dc719b7-589b769fb-244xm\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:28.044714 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.044682    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"]
Sep 11 14:13:28.048402 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.048382    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:28.052609 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.052587    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1dc719b7-dockercfg-8lvg5\""
Sep 11 14:13:28.053117 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.053099    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1dc719b7\""
Sep 11 14:13:28.060021 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.060000    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"]
Sep 11 14:13:28.168219 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.168186    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:28.168428 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.168405    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f6971c6-0f6a-4053-becc-f17ce97426d8-plugins-config-volume\") pod \"payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5\" (UID: \"9f6971c6-0f6a-4053-becc-f17ce97426d8\") " pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:28.168473 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.168448    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7kvn\" (UniqueName: \"kubernetes.io/projected/9f6971c6-0f6a-4053-becc-f17ce97426d8-kube-api-access-z7kvn\") pod \"payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5\" (UID: \"9f6971c6-0f6a-4053-becc-f17ce97426d8\") " pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:28.268976 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.268946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f6971c6-0f6a-4053-becc-f17ce97426d8-plugins-config-volume\") pod \"payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5\" (UID: \"9f6971c6-0f6a-4053-becc-f17ce97426d8\") " pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:28.269128 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.268990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7kvn\" (UniqueName: \"kubernetes.io/projected/9f6971c6-0f6a-4053-becc-f17ce97426d8-kube-api-access-z7kvn\") pod \"payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5\" (UID: \"9f6971c6-0f6a-4053-becc-f17ce97426d8\") " pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:28.269681 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.269663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f6971c6-0f6a-4053-becc-f17ce97426d8-plugins-config-volume\") pod \"payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5\" (UID: \"9f6971c6-0f6a-4053-becc-f17ce97426d8\") " pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:28.283983 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.283959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7kvn\" (UniqueName: \"kubernetes.io/projected/9f6971c6-0f6a-4053-becc-f17ce97426d8-kube-api-access-z7kvn\") pod \"payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5\" (UID: \"9f6971c6-0f6a-4053-becc-f17ce97426d8\") " pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:28.358971 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.358940    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:28.451544 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.451511    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"]
Sep 11 14:13:28.453135 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:13:28.453103    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbee1e10_fab3_4fa5_ac08_cca21ab16d61.slice/crio-eb7785cda3830410548145530537f68c198a2aa5f552f949efbabe8e92acbb38 WatchSource:0}: Error finding container eb7785cda3830410548145530537f68c198a2aa5f552f949efbabe8e92acbb38: Status 404 returned error can't find the container with id eb7785cda3830410548145530537f68c198a2aa5f552f949efbabe8e92acbb38
Sep 11 14:13:28.633489 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:28.633458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"]
Sep 11 14:13:28.634611 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:13:28.634582    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6971c6_0f6a_4053_becc_f17ce97426d8.slice/crio-0ed7c67bfe6292129fd84123baf628f03c4b33836a64e4832cced54d8cbd85da WatchSource:0}: Error finding container 0ed7c67bfe6292129fd84123baf628f03c4b33836a64e4832cced54d8cbd85da: Status 404 returned error can't find the container with id 0ed7c67bfe6292129fd84123baf628f03c4b33836a64e4832cced54d8cbd85da
Sep 11 14:13:29.371488 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.371447    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5" event={"ID":"9f6971c6-0f6a-4053-becc-f17ce97426d8","Type":"ContainerStarted","Data":"0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9"}
Sep 11 14:13:29.371710 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.371581    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:29.371710 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.371641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5" event={"ID":"9f6971c6-0f6a-4053-becc-f17ce97426d8","Type":"ContainerStarted","Data":"0ed7c67bfe6292129fd84123baf628f03c4b33836a64e4832cced54d8cbd85da"}
Sep 11 14:13:29.373332 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.373307    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm" event={"ID":"fbee1e10-fab3-4fa5-ac08-cca21ab16d61","Type":"ContainerStarted","Data":"1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10"}
Sep 11 14:13:29.373427 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.373353    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm" event={"ID":"fbee1e10-fab3-4fa5-ac08-cca21ab16d61","Type":"ContainerStarted","Data":"eb7785cda3830410548145530537f68c198a2aa5f552f949efbabe8e92acbb38"}
Sep 11 14:13:29.373470 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.373430    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:29.419928 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.419873    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5" podStartSLOduration=1.419854948 podStartE2EDuration="1.419854948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:29.41294058 +0000 UTC m=+959.750516140" watchObservedRunningTime="2026-09-11 14:13:29.419854948 +0000 UTC m=+959.757430508"
Sep 11 14:13:29.423912 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.423883    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"]
Sep 11 14:13:29.427317 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.427300    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:29.430290 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.430266    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-2a21d9-dockercfg-hc8nl\""
Sep 11 14:13:29.430668 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.430646    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-2a21d9\""
Sep 11 14:13:29.464363 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.464332    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"]
Sep 11 14:13:29.500500 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.500472    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"]
Sep 11 14:13:29.503976 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.503956    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:29.527906 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.527881    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"]
Sep 11 14:13:29.539703 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.536184    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm" podStartSLOduration=2.2529279669999998 podStartE2EDuration="2.536163163s" totalImagesPullingTime="283.235196ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:29.531012205 +0000 UTC m=+959.868587767" watchObservedRunningTime="2026-09-11 14:13:29.536163163 +0000 UTC m=+959.873738723"
Sep 11 14:13:29.581383 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.581339    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btplj\" (UniqueName: \"kubernetes.io/projected/32a72900-c20d-46ea-83a5-20460dcee57b-kube-api-access-btplj\") pod \"payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc\" (UID: \"32a72900-c20d-46ea-83a5-20460dcee57b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:29.581522 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.581401    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72rtl\" (UniqueName: \"kubernetes.io/projected/3d31509c-619b-4b79-ae74-d728881d0698-kube-api-access-72rtl\") pod \"payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958\" (UID: \"3d31509c-619b-4b79-ae74-d728881d0698\") " pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:29.581522 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.581466    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a72900-c20d-46ea-83a5-20460dcee57b-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc\" (UID: \"32a72900-c20d-46ea-83a5-20460dcee57b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:29.581679 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.581560    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d31509c-619b-4b79-ae74-d728881d0698-plugins-config-volume\") pod \"payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958\" (UID: \"3d31509c-619b-4b79-ae74-d728881d0698\") " pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:29.683144 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.683072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btplj\" (UniqueName: \"kubernetes.io/projected/32a72900-c20d-46ea-83a5-20460dcee57b-kube-api-access-btplj\") pod \"payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc\" (UID: \"32a72900-c20d-46ea-83a5-20460dcee57b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:29.683144 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.683118    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72rtl\" (UniqueName: \"kubernetes.io/projected/3d31509c-619b-4b79-ae74-d728881d0698-kube-api-access-72rtl\") pod \"payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958\" (UID: \"3d31509c-619b-4b79-ae74-d728881d0698\") " pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:29.683363 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.683154    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a72900-c20d-46ea-83a5-20460dcee57b-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc\" (UID: \"32a72900-c20d-46ea-83a5-20460dcee57b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:29.683363 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.683187    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d31509c-619b-4b79-ae74-d728881d0698-plugins-config-volume\") pod \"payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958\" (UID: \"3d31509c-619b-4b79-ae74-d728881d0698\") " pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:29.684110 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.684084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a72900-c20d-46ea-83a5-20460dcee57b-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc\" (UID: \"32a72900-c20d-46ea-83a5-20460dcee57b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:29.684179 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.684089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d31509c-619b-4b79-ae74-d728881d0698-plugins-config-volume\") pod \"payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958\" (UID: \"3d31509c-619b-4b79-ae74-d728881d0698\") " pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:29.718374 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.718341    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72rtl\" (UniqueName: \"kubernetes.io/projected/3d31509c-619b-4b79-ae74-d728881d0698-kube-api-access-72rtl\") pod \"payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958\" (UID: \"3d31509c-619b-4b79-ae74-d728881d0698\") " pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:29.718374 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.718372    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btplj\" (UniqueName: \"kubernetes.io/projected/32a72900-c20d-46ea-83a5-20460dcee57b-kube-api-access-btplj\") pod \"payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc\" (UID: \"32a72900-c20d-46ea-83a5-20460dcee57b\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:29.738510 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.738487    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:29.814114 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:29.814088    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:30.015662 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.015622    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"]
Sep 11 14:13:30.018007 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:13:30.017975    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d31509c_619b_4b79_ae74_d728881d0698.slice/crio-fc8bb1b28adf00ea8355d889d340a9b59144b47fe240ee153ce682fa555bef96 WatchSource:0}: Error finding container fc8bb1b28adf00ea8355d889d340a9b59144b47fe240ee153ce682fa555bef96: Status 404 returned error can't find the container with id fc8bb1b28adf00ea8355d889d340a9b59144b47fe240ee153ce682fa555bef96
Sep 11 14:13:30.102951 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.102930    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"]
Sep 11 14:13:30.105216 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:13:30.105183    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32a72900_c20d_46ea_83a5_20460dcee57b.slice/crio-53d6e3e0ab964cc7277541d4d646dc66ae2bbf45443b9a8d01e9af97c98a8404 WatchSource:0}: Error finding container 53d6e3e0ab964cc7277541d4d646dc66ae2bbf45443b9a8d01e9af97c98a8404: Status 404 returned error can't find the container with id 53d6e3e0ab964cc7277541d4d646dc66ae2bbf45443b9a8d01e9af97c98a8404
Sep 11 14:13:30.381160 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.381124    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc" event={"ID":"32a72900-c20d-46ea-83a5-20460dcee57b","Type":"ContainerStarted","Data":"6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32"}
Sep 11 14:13:30.381353 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.381206    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:30.381353 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.381231    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc" event={"ID":"32a72900-c20d-46ea-83a5-20460dcee57b","Type":"ContainerStarted","Data":"53d6e3e0ab964cc7277541d4d646dc66ae2bbf45443b9a8d01e9af97c98a8404"}
Sep 11 14:13:30.383061 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.383030    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958" event={"ID":"3d31509c-619b-4b79-ae74-d728881d0698","Type":"ContainerStarted","Data":"bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2"}
Sep 11 14:13:30.383212 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.383075    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958" event={"ID":"3d31509c-619b-4b79-ae74-d728881d0698","Type":"ContainerStarted","Data":"fc8bb1b28adf00ea8355d889d340a9b59144b47fe240ee153ce682fa555bef96"}
Sep 11 14:13:30.383278 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.383247    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:30.402175 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.402127    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc" podStartSLOduration=1.402113702 podStartE2EDuration="1.402113702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:30.401568265 +0000 UTC m=+960.739143823" watchObservedRunningTime="2026-09-11 14:13:30.402113702 +0000 UTC m=+960.739689262"
Sep 11 14:13:30.422584 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:30.422546    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958" podStartSLOduration=1.422533633 podStartE2EDuration="1.422533633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:30.421707218 +0000 UTC m=+960.759282790" watchObservedRunningTime="2026-09-11 14:13:30.422533633 +0000 UTC m=+960.760109230"
Sep 11 14:13:31.332796 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:31.332764    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:13:31.333251 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:31.332878    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:13:35.389030 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:35.389002    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:40.385282 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:40.385248    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:41.079736 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.079704    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"]
Sep 11 14:13:41.083747 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.083722    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:41.088915 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.088898    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c6156e\""
Sep 11 14:13:41.095121 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.095095    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfca453-a275-465b-94fa-facb441835b1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z\" (UID: \"3cfca453-a275-465b-94fa-facb441835b1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:41.095219 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.095158    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfqvf\" (UniqueName: \"kubernetes.io/projected/3cfca453-a275-465b-94fa-facb441835b1-kube-api-access-wfqvf\") pod \"payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z\" (UID: \"3cfca453-a275-465b-94fa-facb441835b1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:41.117613 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.117591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"]
Sep 11 14:13:41.195564 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.195532    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfca453-a275-465b-94fa-facb441835b1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z\" (UID: \"3cfca453-a275-465b-94fa-facb441835b1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:41.195743 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.195606    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfqvf\" (UniqueName: \"kubernetes.io/projected/3cfca453-a275-465b-94fa-facb441835b1-kube-api-access-wfqvf\") pod \"payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z\" (UID: \"3cfca453-a275-465b-94fa-facb441835b1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:41.196215 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.196197    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfca453-a275-465b-94fa-facb441835b1-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z\" (UID: \"3cfca453-a275-465b-94fa-facb441835b1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:41.207374 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.207348    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfqvf\" (UniqueName: \"kubernetes.io/projected/3cfca453-a275-465b-94fa-facb441835b1-kube-api-access-wfqvf\") pod \"payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z\" (UID: \"3cfca453-a275-465b-94fa-facb441835b1\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:41.391404 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.391332    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:13:41.391772 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.391490    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:13:41.394135 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.394094    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:41.702571 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:41.702492    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"]
Sep 11 14:13:41.705394 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:13:41.705369    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cfca453_a275_465b_94fa_facb441835b1.slice/crio-c8bba5c19ad6203ae860901fd9ac819637f32fbd7d222cbaed3f383edad0936a WatchSource:0}: Error finding container c8bba5c19ad6203ae860901fd9ac819637f32fbd7d222cbaed3f383edad0936a: Status 404 returned error can't find the container with id c8bba5c19ad6203ae860901fd9ac819637f32fbd7d222cbaed3f383edad0936a
Sep 11 14:13:42.447613 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:42.447577    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z" event={"ID":"3cfca453-a275-465b-94fa-facb441835b1","Type":"ContainerStarted","Data":"4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62"}
Sep 11 14:13:42.448102 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:42.447644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z" event={"ID":"3cfca453-a275-465b-94fa-facb441835b1","Type":"ContainerStarted","Data":"c8bba5c19ad6203ae860901fd9ac819637f32fbd7d222cbaed3f383edad0936a"}
Sep 11 14:13:42.448102 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:42.447766    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:42.468689 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:42.468618    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z" podStartSLOduration=1.468599818 podStartE2EDuration="1.468599818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:13:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:13:42.466641528 +0000 UTC m=+972.804217086" watchObservedRunningTime="2026-09-11 14:13:42.468599818 +0000 UTC m=+972.806175382"
Sep 11 14:13:53.456244 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:53.456157    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:13:57.697643 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:57.697592    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"]
Sep 11 14:13:57.698046 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:57.697880    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm" podUID="fbee1e10-fab3-4fa5-ac08-cca21ab16d61" containerName="maas-api" containerID="cri-o://1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10" gracePeriod=30
Sep 11 14:13:57.785148 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:57.785107    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"]
Sep 11 14:13:57.785347 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:57.785316    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5" podUID="9f6971c6-0f6a-4053-becc-f17ce97426d8" containerName="payload-processing" containerID="cri-o://0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9" gracePeriod=30
Sep 11 14:13:58.034638 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.034607    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:58.067245 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.067222    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:58.151228 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.151196    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7kvn\" (UniqueName: \"kubernetes.io/projected/9f6971c6-0f6a-4053-becc-f17ce97426d8-kube-api-access-z7kvn\") pod \"9f6971c6-0f6a-4053-becc-f17ce97426d8\" (UID: \"9f6971c6-0f6a-4053-becc-f17ce97426d8\") "
Sep 11 14:13:58.151228 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.151230    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f6971c6-0f6a-4053-becc-f17ce97426d8-plugins-config-volume\") pod \"9f6971c6-0f6a-4053-becc-f17ce97426d8\" (UID: \"9f6971c6-0f6a-4053-becc-f17ce97426d8\") "
Sep 11 14:13:58.151424 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.151288    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-metrics-certs\") pod \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") "
Sep 11 14:13:58.151424 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.151329    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nf7p\" (UniqueName: \"kubernetes.io/projected/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-kube-api-access-8nf7p\") pod \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") "
Sep 11 14:13:58.151424 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.151379    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-openshift-service-ca\") pod \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") "
Sep 11 14:13:58.151424 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.151404    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-maas-api-tls\") pod \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\" (UID: \"fbee1e10-fab3-4fa5-ac08-cca21ab16d61\") "
Sep 11 14:13:58.151771 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.151734    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f6971c6-0f6a-4053-becc-f17ce97426d8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9f6971c6-0f6a-4053-becc-f17ce97426d8" (UID: "9f6971c6-0f6a-4053-becc-f17ce97426d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:13:58.151873 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.151780    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fbee1e10-fab3-4fa5-ac08-cca21ab16d61" (UID: "fbee1e10-fab3-4fa5-ac08-cca21ab16d61"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:13:58.153573 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.153546    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-kube-api-access-8nf7p" (OuterVolumeSpecName: "kube-api-access-8nf7p") pod "fbee1e10-fab3-4fa5-ac08-cca21ab16d61" (UID: "fbee1e10-fab3-4fa5-ac08-cca21ab16d61"). InnerVolumeSpecName "kube-api-access-8nf7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:58.153686 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.153572    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fbee1e10-fab3-4fa5-ac08-cca21ab16d61" (UID: "fbee1e10-fab3-4fa5-ac08-cca21ab16d61"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:13:58.153686 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.153572    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f6971c6-0f6a-4053-becc-f17ce97426d8-kube-api-access-z7kvn" (OuterVolumeSpecName: "kube-api-access-z7kvn") pod "9f6971c6-0f6a-4053-becc-f17ce97426d8" (UID: "9f6971c6-0f6a-4053-becc-f17ce97426d8"). InnerVolumeSpecName "kube-api-access-z7kvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:13:58.153686 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.153597    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fbee1e10-fab3-4fa5-ac08-cca21ab16d61" (UID: "fbee1e10-fab3-4fa5-ac08-cca21ab16d61"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:13:58.252544 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.252455    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7kvn\" (UniqueName: \"kubernetes.io/projected/9f6971c6-0f6a-4053-becc-f17ce97426d8-kube-api-access-z7kvn\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:13:58.252544 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.252487    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9f6971c6-0f6a-4053-becc-f17ce97426d8-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:13:58.252544 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.252501    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:13:58.252544 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.252513    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nf7p\" (UniqueName: \"kubernetes.io/projected/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-kube-api-access-8nf7p\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:13:58.252544 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.252525    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:13:58.252544 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.252537    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbee1e10-fab3-4fa5-ac08-cca21ab16d61-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:13:58.534962 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.534881    2564 generic.go:356] "Generic (PLEG): container finished" podID="9f6971c6-0f6a-4053-becc-f17ce97426d8" containerID="0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9" exitCode=0
Sep 11 14:13:58.534962 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.534927    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"
Sep 11 14:13:58.535154 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.534973    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5" event={"ID":"9f6971c6-0f6a-4053-becc-f17ce97426d8","Type":"ContainerDied","Data":"0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9"}
Sep 11 14:13:58.535154 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.535025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5" event={"ID":"9f6971c6-0f6a-4053-becc-f17ce97426d8","Type":"ContainerDied","Data":"0ed7c67bfe6292129fd84123baf628f03c4b33836a64e4832cced54d8cbd85da"}
Sep 11 14:13:58.535154 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.535055    2564 scope.go:117] "RemoveContainer" containerID="0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9"
Sep 11 14:13:58.536548 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.536525    2564 generic.go:356] "Generic (PLEG): container finished" podID="fbee1e10-fab3-4fa5-ac08-cca21ab16d61" containerID="1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10" exitCode=0
Sep 11 14:13:58.536722 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.536564    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"
Sep 11 14:13:58.536722 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.536560    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm" event={"ID":"fbee1e10-fab3-4fa5-ac08-cca21ab16d61","Type":"ContainerDied","Data":"1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10"}
Sep 11 14:13:58.536722 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.536693    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm" event={"ID":"fbee1e10-fab3-4fa5-ac08-cca21ab16d61","Type":"ContainerDied","Data":"eb7785cda3830410548145530537f68c198a2aa5f552f949efbabe8e92acbb38"}
Sep 11 14:13:58.544362 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.544347    2564 scope.go:117] "RemoveContainer" containerID="0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9"
Sep 11 14:13:58.544640 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:13:58.544609    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9\": container with ID starting with 0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9 not found: ID does not exist" containerID="0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9"
Sep 11 14:13:58.544691 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.544651    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9"} err="failed to get container status \"0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9\": rpc error: code = NotFound desc = could not find container \"0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9\": container with ID starting with 0a9e9e320b2f54d4d7b5c0d30be7a914d6a2cadee3aa85bf1255430de06ccca9 not found: ID does not exist"
Sep 11 14:13:58.544691 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.544667    2564 scope.go:117] "RemoveContainer" containerID="1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10"
Sep 11 14:13:58.552411 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.552389    2564 scope.go:117] "RemoveContainer" containerID="1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10"
Sep 11 14:13:58.552693 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:13:58.552674    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10\": container with ID starting with 1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10 not found: ID does not exist" containerID="1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10"
Sep 11 14:13:58.552760 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.552703    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10"} err="failed to get container status \"1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10\": rpc error: code = NotFound desc = could not find container \"1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10\": container with ID starting with 1f059e94990f1e022eab23896607a2e63e9de9dbd528533c58ddab9244ac6a10 not found: ID does not exist"
Sep 11 14:13:58.557723 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.557702    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"]
Sep 11 14:13:58.564719 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.564700    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-1dc719b7-589b769fb-244xm"]
Sep 11 14:13:58.578924 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.578900    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"]
Sep 11 14:13:58.583601 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:13:58.583581    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1dc719b7-b57689b7f-zkbb5"]
Sep 11 14:14:00.275430 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:00.275393    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f6971c6-0f6a-4053-becc-f17ce97426d8" path="/var/lib/kubelet/pods/9f6971c6-0f6a-4053-becc-f17ce97426d8/volumes"
Sep 11 14:14:00.276001 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:00.275979    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbee1e10-fab3-4fa5-ac08-cca21ab16d61" path="/var/lib/kubelet/pods/fbee1e10-fab3-4fa5-ac08-cca21ab16d61/volumes"
Sep 11 14:14:08.392443 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.392404    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"]
Sep 11 14:14:08.392922 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.392880    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbee1e10-fab3-4fa5-ac08-cca21ab16d61" containerName="maas-api"
Sep 11 14:14:08.392922 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.392898    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbee1e10-fab3-4fa5-ac08-cca21ab16d61" containerName="maas-api"
Sep 11 14:14:08.392922 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.392917    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f6971c6-0f6a-4053-becc-f17ce97426d8" containerName="payload-processing"
Sep 11 14:14:08.392922 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.392923    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f6971c6-0f6a-4053-becc-f17ce97426d8" containerName="payload-processing"
Sep 11 14:14:08.393065 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.392998    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbee1e10-fab3-4fa5-ac08-cca21ab16d61" containerName="maas-api"
Sep 11 14:14:08.393065 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.393006    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f6971c6-0f6a-4053-becc-f17ce97426d8" containerName="payload-processing"
Sep 11 14:14:08.397902 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.397880    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:08.402209 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.402187    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-59484b3d-dockercfg-g6cqt\""
Sep 11 14:14:08.402890 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.402875    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-59484b3d\""
Sep 11 14:14:08.412641 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.412604    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"]
Sep 11 14:14:08.546457 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.546421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7wbw\" (UniqueName: \"kubernetes.io/projected/c311ba38-962f-4a8f-a847-36ab66b422c2-kube-api-access-k7wbw\") pod \"payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8\" (UID: \"c311ba38-962f-4a8f-a847-36ab66b422c2\") " pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:08.546457 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.546459    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c311ba38-962f-4a8f-a847-36ab66b422c2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8\" (UID: \"c311ba38-962f-4a8f-a847-36ab66b422c2\") " pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:08.647454 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.647376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7wbw\" (UniqueName: \"kubernetes.io/projected/c311ba38-962f-4a8f-a847-36ab66b422c2-kube-api-access-k7wbw\") pod \"payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8\" (UID: \"c311ba38-962f-4a8f-a847-36ab66b422c2\") " pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:08.647454 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.647410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c311ba38-962f-4a8f-a847-36ab66b422c2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8\" (UID: \"c311ba38-962f-4a8f-a847-36ab66b422c2\") " pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:08.648125 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.648107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c311ba38-962f-4a8f-a847-36ab66b422c2-plugins-config-volume\") pod \"payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8\" (UID: \"c311ba38-962f-4a8f-a847-36ab66b422c2\") " pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:08.671191 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.671162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7wbw\" (UniqueName: \"kubernetes.io/projected/c311ba38-962f-4a8f-a847-36ab66b422c2-kube-api-access-k7wbw\") pod \"payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8\" (UID: \"c311ba38-962f-4a8f-a847-36ab66b422c2\") " pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:08.708205 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:08.708178    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:09.007815 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:09.007745    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"]
Sep 11 14:14:09.011719 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:14:09.011695    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc311ba38_962f_4a8f_a847_36ab66b422c2.slice/crio-18703ab93414c2cf48229cca658d8b77d7e316454b154b383675295be4669d98 WatchSource:0}: Error finding container 18703ab93414c2cf48229cca658d8b77d7e316454b154b383675295be4669d98: Status 404 returned error can't find the container with id 18703ab93414c2cf48229cca658d8b77d7e316454b154b383675295be4669d98
Sep 11 14:14:09.597036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:09.597000    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8" event={"ID":"c311ba38-962f-4a8f-a847-36ab66b422c2","Type":"ContainerStarted","Data":"7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df"}
Sep 11 14:14:09.597497 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:09.597056    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8" event={"ID":"c311ba38-962f-4a8f-a847-36ab66b422c2","Type":"ContainerStarted","Data":"18703ab93414c2cf48229cca658d8b77d7e316454b154b383675295be4669d98"}
Sep 11 14:14:09.597497 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:09.597146    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:20.606825 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:20.606793    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:20.633726 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:20.633678    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8" podStartSLOduration=12.633662259 podStartE2EDuration="12.633662259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:14:09.621962905 +0000 UTC m=+999.959538466" watchObservedRunningTime="2026-09-11 14:14:20.633662259 +0000 UTC m=+1010.971237818"
Sep 11 14:14:45.500305 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.500268    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"]
Sep 11 14:14:45.500674 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.500458    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8" podUID="c311ba38-962f-4a8f-a847-36ab66b422c2" containerName="payload-processing" containerID="cri-o://7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df" gracePeriod=30
Sep 11 14:14:45.754782 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.754723    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:45.779275 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.779244    2564 generic.go:356] "Generic (PLEG): container finished" podID="c311ba38-962f-4a8f-a847-36ab66b422c2" containerID="7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df" exitCode=0
Sep 11 14:14:45.779404 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.779284    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"
Sep 11 14:14:45.779404 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.779304    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8" event={"ID":"c311ba38-962f-4a8f-a847-36ab66b422c2","Type":"ContainerDied","Data":"7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df"}
Sep 11 14:14:45.779404 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.779378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8" event={"ID":"c311ba38-962f-4a8f-a847-36ab66b422c2","Type":"ContainerDied","Data":"18703ab93414c2cf48229cca658d8b77d7e316454b154b383675295be4669d98"}
Sep 11 14:14:45.779404 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.779400    2564 scope.go:117] "RemoveContainer" containerID="7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df"
Sep 11 14:14:45.788902 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.788809    2564 scope.go:117] "RemoveContainer" containerID="7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df"
Sep 11 14:14:45.789167 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:14:45.789144    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df\": container with ID starting with 7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df not found: ID does not exist" containerID="7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df"
Sep 11 14:14:45.789244 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.789180    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df"} err="failed to get container status \"7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df\": rpc error: code = NotFound desc = could not find container \"7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df\": container with ID starting with 7936f8d37eda6de088975bac07505103f3262afbd64bac01df131a3cce5052df not found: ID does not exist"
Sep 11 14:14:45.889308 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.889276    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7wbw\" (UniqueName: \"kubernetes.io/projected/c311ba38-962f-4a8f-a847-36ab66b422c2-kube-api-access-k7wbw\") pod \"c311ba38-962f-4a8f-a847-36ab66b422c2\" (UID: \"c311ba38-962f-4a8f-a847-36ab66b422c2\") "
Sep 11 14:14:45.889489 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.889449    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c311ba38-962f-4a8f-a847-36ab66b422c2-plugins-config-volume\") pod \"c311ba38-962f-4a8f-a847-36ab66b422c2\" (UID: \"c311ba38-962f-4a8f-a847-36ab66b422c2\") "
Sep 11 14:14:45.889882 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.889855    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c311ba38-962f-4a8f-a847-36ab66b422c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c311ba38-962f-4a8f-a847-36ab66b422c2" (UID: "c311ba38-962f-4a8f-a847-36ab66b422c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:14:45.891566 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.891542    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c311ba38-962f-4a8f-a847-36ab66b422c2-kube-api-access-k7wbw" (OuterVolumeSpecName: "kube-api-access-k7wbw") pod "c311ba38-962f-4a8f-a847-36ab66b422c2" (UID: "c311ba38-962f-4a8f-a847-36ab66b422c2"). InnerVolumeSpecName "kube-api-access-k7wbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:14:45.991167 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.991133    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c311ba38-962f-4a8f-a847-36ab66b422c2-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:14:45.991167 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:45.991164    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7wbw\" (UniqueName: \"kubernetes.io/projected/c311ba38-962f-4a8f-a847-36ab66b422c2-kube-api-access-k7wbw\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:14:46.104743 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:46.104718    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"]
Sep 11 14:14:46.111032 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:46.111009    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-59484b3d-654f7fb486-wrfk8"]
Sep 11 14:14:46.276200 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:46.276170    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c311ba38-962f-4a8f-a847-36ab66b422c2" path="/var/lib/kubelet/pods/c311ba38-962f-4a8f-a847-36ab66b422c2/volumes"
Sep 11 14:14:52.206038 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.205998    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"]
Sep 11 14:14:52.206992 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.206604    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c311ba38-962f-4a8f-a847-36ab66b422c2" containerName="payload-processing"
Sep 11 14:14:52.206992 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.206622    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c311ba38-962f-4a8f-a847-36ab66b422c2" containerName="payload-processing"
Sep 11 14:14:52.206992 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.206753    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c311ba38-962f-4a8f-a847-36ab66b422c2" containerName="payload-processing"
Sep 11 14:14:52.211215 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.211191    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:14:52.214021 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.214001    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-65bba332\""
Sep 11 14:14:52.225217 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.224590    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"]
Sep 11 14:14:52.352393 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.352354    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b9hd\" (UniqueName: \"kubernetes.io/projected/596a590d-5c03-43d2-8b65-4892fd8f7715-kube-api-access-7b9hd\") pod \"payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb\" (UID: \"596a590d-5c03-43d2-8b65-4892fd8f7715\") " pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:14:52.352733 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.352706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596a590d-5c03-43d2-8b65-4892fd8f7715-plugins-config-volume\") pod \"payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb\" (UID: \"596a590d-5c03-43d2-8b65-4892fd8f7715\") " pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:14:52.453341 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.453306    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b9hd\" (UniqueName: \"kubernetes.io/projected/596a590d-5c03-43d2-8b65-4892fd8f7715-kube-api-access-7b9hd\") pod \"payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb\" (UID: \"596a590d-5c03-43d2-8b65-4892fd8f7715\") " pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:14:52.453524 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.453416    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596a590d-5c03-43d2-8b65-4892fd8f7715-plugins-config-volume\") pod \"payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb\" (UID: \"596a590d-5c03-43d2-8b65-4892fd8f7715\") " pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:14:52.454156 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.454135    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596a590d-5c03-43d2-8b65-4892fd8f7715-plugins-config-volume\") pod \"payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb\" (UID: \"596a590d-5c03-43d2-8b65-4892fd8f7715\") " pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:14:52.462063 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.462004    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b9hd\" (UniqueName: \"kubernetes.io/projected/596a590d-5c03-43d2-8b65-4892fd8f7715-kube-api-access-7b9hd\") pod \"payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb\" (UID: \"596a590d-5c03-43d2-8b65-4892fd8f7715\") " pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:14:52.529481 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.529451    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:14:52.807593 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:52.807562    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"]
Sep 11 14:14:52.811530 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:14:52.811497    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596a590d_5c03_43d2_8b65_4892fd8f7715.slice/crio-fee4c35adf4e88ddbb81abdf61e6fdde776b0d43680ad8e31aac0bf05c04dd3a WatchSource:0}: Error finding container fee4c35adf4e88ddbb81abdf61e6fdde776b0d43680ad8e31aac0bf05c04dd3a: Status 404 returned error can't find the container with id fee4c35adf4e88ddbb81abdf61e6fdde776b0d43680ad8e31aac0bf05c04dd3a
Sep 11 14:14:53.822190 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:53.822151    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb" event={"ID":"596a590d-5c03-43d2-8b65-4892fd8f7715","Type":"ContainerStarted","Data":"88d0851ec30c00fab1263dee9d2d1e4aedd4ce9e31c9a5a5bbaa90d582846315"}
Sep 11 14:14:53.822578 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:53.822206    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb" event={"ID":"596a590d-5c03-43d2-8b65-4892fd8f7715","Type":"ContainerStarted","Data":"fee4c35adf4e88ddbb81abdf61e6fdde776b0d43680ad8e31aac0bf05c04dd3a"}
Sep 11 14:14:53.822578 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:53.822250    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:14:53.845157 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:14:53.845105    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb" podStartSLOduration=1.845088252 podStartE2EDuration="1.845088252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:14:53.843088399 +0000 UTC m=+1044.180663971" watchObservedRunningTime="2026-09-11 14:14:53.845088252 +0000 UTC m=+1044.182663811"
Sep 11 14:15:00.201784 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:00.201752    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth"]
Sep 11 14:15:00.205060 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:00.205036    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth"
Sep 11 14:15:00.208592 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:00.208566    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zrfn2\""
Sep 11 14:15:00.215126 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:00.215106    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth"
Sep 11 14:15:00.233676 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:00.233647    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth"]
Sep 11 14:15:00.515482 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:00.515458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth"]
Sep 11 14:15:00.517551 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:15:00.517524    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7007cff9_b62d_4ac1_9950_bf9b9123455c.slice/crio-90b1eed6eef03a488ada7f6704258575cc5298ba0f7928f3ae85c48938b27faf WatchSource:0}: Error finding container 90b1eed6eef03a488ada7f6704258575cc5298ba0f7928f3ae85c48938b27faf: Status 404 returned error can't find the container with id 90b1eed6eef03a488ada7f6704258575cc5298ba0f7928f3ae85c48938b27faf
Sep 11 14:15:00.519513 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:00.519491    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:15:00.856506 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:00.856475    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth" event={"ID":"7007cff9-b62d-4ac1-9950-bf9b9123455c","Type":"ContainerStarted","Data":"90b1eed6eef03a488ada7f6704258575cc5298ba0f7928f3ae85c48938b27faf"}
Sep 11 14:15:02.867991 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:02.867960    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth" event={"ID":"7007cff9-b62d-4ac1-9950-bf9b9123455c","Type":"ContainerStarted","Data":"1fabb0209744097e83498f5bcb76e5860cfa97f2d9c48d7bde7178d5c51870d1"}
Sep 11 14:15:02.894293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:02.894250    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth" podStartSLOduration=0.820348071 podStartE2EDuration="2.894236022s" totalImagesPullingTime="2.073887951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:15:02.893876627 +0000 UTC m=+1053.231452187" watchObservedRunningTime="2026-09-11 14:15:02.894236022 +0000 UTC m=+1053.231811581"
Sep 11 14:15:03.874216 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:03.874177    2564 generic.go:356] "Generic (PLEG): container finished" podID="7007cff9-b62d-4ac1-9950-bf9b9123455c" containerID="1fabb0209744097e83498f5bcb76e5860cfa97f2d9c48d7bde7178d5c51870d1" exitCode=0
Sep 11 14:15:03.874216 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:03.874221    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth" event={"ID":"7007cff9-b62d-4ac1-9950-bf9b9123455c","Type":"ContainerDied","Data":"1fabb0209744097e83498f5bcb76e5860cfa97f2d9c48d7bde7178d5c51870d1"}
Sep 11 14:15:04.828806 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:04.828777    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:15:05.014844 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:05.014799    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth"
Sep 11 14:15:05.884943 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:05.884911    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth" event={"ID":"7007cff9-b62d-4ac1-9950-bf9b9123455c","Type":"ContainerDied","Data":"90b1eed6eef03a488ada7f6704258575cc5298ba0f7928f3ae85c48938b27faf"}
Sep 11 14:15:05.884943 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:05.884945    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth"
Sep 11 14:15:05.885165 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:05.884955    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="90b1eed6eef03a488ada7f6704258575cc5298ba0f7928f3ae85c48938b27faf"
Sep 11 14:15:21.795208 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:21.795175    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"]
Sep 11 14:15:21.795878 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:21.795435    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb" podUID="596a590d-5c03-43d2-8b65-4892fd8f7715" containerName="payload-processing" containerID="cri-o://88d0851ec30c00fab1263dee9d2d1e4aedd4ce9e31c9a5a5bbaa90d582846315" gracePeriod=30
Sep 11 14:15:21.969557 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:21.969521    2564 generic.go:356] "Generic (PLEG): container finished" podID="596a590d-5c03-43d2-8b65-4892fd8f7715" containerID="88d0851ec30c00fab1263dee9d2d1e4aedd4ce9e31c9a5a5bbaa90d582846315" exitCode=0
Sep 11 14:15:21.969702 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:21.969593    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb" event={"ID":"596a590d-5c03-43d2-8b65-4892fd8f7715","Type":"ContainerDied","Data":"88d0851ec30c00fab1263dee9d2d1e4aedd4ce9e31c9a5a5bbaa90d582846315"}
Sep 11 14:15:22.056600 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.056578    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:15:22.118451 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.118416    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7b9hd\" (UniqueName: \"kubernetes.io/projected/596a590d-5c03-43d2-8b65-4892fd8f7715-kube-api-access-7b9hd\") pod \"596a590d-5c03-43d2-8b65-4892fd8f7715\" (UID: \"596a590d-5c03-43d2-8b65-4892fd8f7715\") "
Sep 11 14:15:22.118606 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.118492    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596a590d-5c03-43d2-8b65-4892fd8f7715-plugins-config-volume\") pod \"596a590d-5c03-43d2-8b65-4892fd8f7715\" (UID: \"596a590d-5c03-43d2-8b65-4892fd8f7715\") "
Sep 11 14:15:22.118912 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.118890    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/596a590d-5c03-43d2-8b65-4892fd8f7715-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "596a590d-5c03-43d2-8b65-4892fd8f7715" (UID: "596a590d-5c03-43d2-8b65-4892fd8f7715"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:15:22.120567 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.120544    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596a590d-5c03-43d2-8b65-4892fd8f7715-kube-api-access-7b9hd" (OuterVolumeSpecName: "kube-api-access-7b9hd") pod "596a590d-5c03-43d2-8b65-4892fd8f7715" (UID: "596a590d-5c03-43d2-8b65-4892fd8f7715"). InnerVolumeSpecName "kube-api-access-7b9hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:15:22.219791 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.219760    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596a590d-5c03-43d2-8b65-4892fd8f7715-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:15:22.219791 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.219787    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7b9hd\" (UniqueName: \"kubernetes.io/projected/596a590d-5c03-43d2-8b65-4892fd8f7715-kube-api-access-7b9hd\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:15:22.976758 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.976687    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"
Sep 11 14:15:22.976758 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.976712    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb" event={"ID":"596a590d-5c03-43d2-8b65-4892fd8f7715","Type":"ContainerDied","Data":"fee4c35adf4e88ddbb81abdf61e6fdde776b0d43680ad8e31aac0bf05c04dd3a"}
Sep 11 14:15:22.977250 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.976771    2564 scope.go:117] "RemoveContainer" containerID="88d0851ec30c00fab1263dee9d2d1e4aedd4ce9e31c9a5a5bbaa90d582846315"
Sep 11 14:15:22.999357 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:22.999327    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"]
Sep 11 14:15:23.006385 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:23.006363    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-65bba332-68445fc5cf-4fzgb"]
Sep 11 14:15:24.275736 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:24.275708    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="596a590d-5c03-43d2-8b65-4892fd8f7715" path="/var/lib/kubelet/pods/596a590d-5c03-43d2-8b65-4892fd8f7715/volumes"
Sep 11 14:15:27.605795 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.605758    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"]
Sep 11 14:15:27.606428 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.606410    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7007cff9-b62d-4ac1-9950-bf9b9123455c" containerName="cleanup"
Sep 11 14:15:27.606516 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.606430    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7007cff9-b62d-4ac1-9950-bf9b9123455c" containerName="cleanup"
Sep 11 14:15:27.606516 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.606461    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="596a590d-5c03-43d2-8b65-4892fd8f7715" containerName="payload-processing"
Sep 11 14:15:27.606516 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.606470    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="596a590d-5c03-43d2-8b65-4892fd8f7715" containerName="payload-processing"
Sep 11 14:15:27.606695 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.606586    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="596a590d-5c03-43d2-8b65-4892fd8f7715" containerName="payload-processing"
Sep 11 14:15:27.606695 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.606604    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7007cff9-b62d-4ac1-9950-bf9b9123455c" containerName="cleanup"
Sep 11 14:15:27.610855 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.610834    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.613293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.613270    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-d609a19a-serving-cert\""
Sep 11 14:15:27.629030 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.629011    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"]
Sep 11 14:15:27.669445 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.669413    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/433f09d8-cef4-4171-ae26-7bf134fe1241-openshift-service-ca\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.669595 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.669497    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-maas-api-tls\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.669595 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.669550    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-metrics-certs\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.669595 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.669574    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbkzm\" (UniqueName: \"kubernetes.io/projected/433f09d8-cef4-4171-ae26-7bf134fe1241-kube-api-access-qbkzm\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.770911 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.770882    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-maas-api-tls\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.771079 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.770919    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-metrics-certs\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.771079 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.770939    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbkzm\" (UniqueName: \"kubernetes.io/projected/433f09d8-cef4-4171-ae26-7bf134fe1241-kube-api-access-qbkzm\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.771079 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.770986    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/433f09d8-cef4-4171-ae26-7bf134fe1241-openshift-service-ca\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.771079 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:15:27.771062    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-serving-cert: secret "maas-api-e2e-derive-d609a19a-serving-cert" not found
Sep 11 14:15:27.771289 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:15:27.771138    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-maas-api-tls podName:433f09d8-cef4-4171-ae26-7bf134fe1241 nodeName:}" failed. No retries permitted until 2026-09-11 14:15:28.2711171 +0000 UTC m=+1078.608692656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-maas-api-tls") pod "maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv" (UID: "433f09d8-cef4-4171-ae26-7bf134fe1241") : secret "maas-api-e2e-derive-d609a19a-serving-cert" not found
Sep 11 14:15:27.771607 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.771590    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/433f09d8-cef4-4171-ae26-7bf134fe1241-openshift-service-ca\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.773425 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.773398    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-metrics-certs\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:27.785702 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:27.785678    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbkzm\" (UniqueName: \"kubernetes.io/projected/433f09d8-cef4-4171-ae26-7bf134fe1241-kube-api-access-qbkzm\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:28.274891 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:28.274855    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-maas-api-tls\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:28.277575 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:28.277542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-maas-api-tls\") pod \"maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:28.520507 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:28.520477    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:28.801739 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:28.801715    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"]
Sep 11 14:15:28.803880 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:15:28.803856    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod433f09d8_cef4_4171_ae26_7bf134fe1241.slice/crio-960212c5c3a3224479d50a53c4bfb25ddef3b4195d72c7827d8b2f8ebfedd29b WatchSource:0}: Error finding container 960212c5c3a3224479d50a53c4bfb25ddef3b4195d72c7827d8b2f8ebfedd29b: Status 404 returned error can't find the container with id 960212c5c3a3224479d50a53c4bfb25ddef3b4195d72c7827d8b2f8ebfedd29b
Sep 11 14:15:29.010866 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:29.010831    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv" event={"ID":"433f09d8-cef4-4171-ae26-7bf134fe1241","Type":"ContainerStarted","Data":"960212c5c3a3224479d50a53c4bfb25ddef3b4195d72c7827d8b2f8ebfedd29b"}
Sep 11 14:15:30.018024 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:30.017983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv" event={"ID":"433f09d8-cef4-4171-ae26-7bf134fe1241","Type":"ContainerStarted","Data":"5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8"}
Sep 11 14:15:30.018498 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:30.018128    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:30.042974 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:30.042925    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv" podStartSLOduration=2.625072495 podStartE2EDuration="3.042909729s" totalImagesPullingTime="417.837234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:15:30.041464069 +0000 UTC m=+1080.379039641" watchObservedRunningTime="2026-09-11 14:15:30.042909729 +0000 UTC m=+1080.380485288"
Sep 11 14:15:36.030410 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:36.030378    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:59.488705 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.488673    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"]
Sep 11 14:15:59.489251 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.488899    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv" podUID="433f09d8-cef4-4171-ae26-7bf134fe1241" containerName="maas-api" containerID="cri-o://5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8" gracePeriod=30
Sep 11 14:15:59.750849 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.750795    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:15:59.853729 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.853696    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbkzm\" (UniqueName: \"kubernetes.io/projected/433f09d8-cef4-4171-ae26-7bf134fe1241-kube-api-access-qbkzm\") pod \"433f09d8-cef4-4171-ae26-7bf134fe1241\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") "
Sep 11 14:15:59.853915 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.853752    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-maas-api-tls\") pod \"433f09d8-cef4-4171-ae26-7bf134fe1241\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") "
Sep 11 14:15:59.853915 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.853826    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/433f09d8-cef4-4171-ae26-7bf134fe1241-openshift-service-ca\") pod \"433f09d8-cef4-4171-ae26-7bf134fe1241\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") "
Sep 11 14:15:59.853915 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.853854    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-metrics-certs\") pod \"433f09d8-cef4-4171-ae26-7bf134fe1241\" (UID: \"433f09d8-cef4-4171-ae26-7bf134fe1241\") "
Sep 11 14:15:59.854296 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.854268    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/433f09d8-cef4-4171-ae26-7bf134fe1241-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "433f09d8-cef4-4171-ae26-7bf134fe1241" (UID: "433f09d8-cef4-4171-ae26-7bf134fe1241"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:15:59.855921 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.855890    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/433f09d8-cef4-4171-ae26-7bf134fe1241-kube-api-access-qbkzm" (OuterVolumeSpecName: "kube-api-access-qbkzm") pod "433f09d8-cef4-4171-ae26-7bf134fe1241" (UID: "433f09d8-cef4-4171-ae26-7bf134fe1241"). InnerVolumeSpecName "kube-api-access-qbkzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:15:59.856023 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.855920    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "433f09d8-cef4-4171-ae26-7bf134fe1241" (UID: "433f09d8-cef4-4171-ae26-7bf134fe1241"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:15:59.856023 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.855939    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "433f09d8-cef4-4171-ae26-7bf134fe1241" (UID: "433f09d8-cef4-4171-ae26-7bf134fe1241"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:15:59.955530 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.955497    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbkzm\" (UniqueName: \"kubernetes.io/projected/433f09d8-cef4-4171-ae26-7bf134fe1241-kube-api-access-qbkzm\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.955530 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.955527    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.955530 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.955538    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/433f09d8-cef4-4171-ae26-7bf134fe1241-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:15:59.955791 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:15:59.955547    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/433f09d8-cef4-4171-ae26-7bf134fe1241-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:16:00.169561 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.169526    2564 generic.go:356] "Generic (PLEG): container finished" podID="433f09d8-cef4-4171-ae26-7bf134fe1241" containerID="5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8" exitCode=0
Sep 11 14:16:00.169742 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.169614    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"
Sep 11 14:16:00.169742 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.169612    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv" event={"ID":"433f09d8-cef4-4171-ae26-7bf134fe1241","Type":"ContainerDied","Data":"5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8"}
Sep 11 14:16:00.169832 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.169792    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv" event={"ID":"433f09d8-cef4-4171-ae26-7bf134fe1241","Type":"ContainerDied","Data":"960212c5c3a3224479d50a53c4bfb25ddef3b4195d72c7827d8b2f8ebfedd29b"}
Sep 11 14:16:00.169832 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.169816    2564 scope.go:117] "RemoveContainer" containerID="5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8"
Sep 11 14:16:00.180588 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.180566    2564 scope.go:117] "RemoveContainer" containerID="5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8"
Sep 11 14:16:00.180984 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:16:00.180962    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8\": container with ID starting with 5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8 not found: ID does not exist" containerID="5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8"
Sep 11 14:16:00.181070 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.180996    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8"} err="failed to get container status \"5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8\": rpc error: code = NotFound desc = could not find container \"5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8\": container with ID starting with 5a63c29fd90f538de18eff7b1d205e9e1c920ada196bbab069e18efc7ae2a7d8 not found: ID does not exist"
Sep 11 14:16:00.221012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.220983    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"]
Sep 11 14:16:00.232860 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.232836    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-d609a19a-55fd8fbf74-qx6zv"]
Sep 11 14:16:00.277464 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:00.277425    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="433f09d8-cef4-4171-ae26-7bf134fe1241" path="/var/lib/kubelet/pods/433f09d8-cef4-4171-ae26-7bf134fe1241/volumes"
Sep 11 14:16:04.342527 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.342490    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"]
Sep 11 14:16:04.342941 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.342807    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958" podUID="3d31509c-619b-4b79-ae74-d728881d0698" containerName="payload-processing" containerID="cri-o://bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2" gracePeriod=30
Sep 11 14:16:04.374880 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.374849    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"]
Sep 11 14:16:04.375129 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.375096    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc" podUID="32a72900-c20d-46ea-83a5-20460dcee57b" containerName="payload-pre-processing" containerID="cri-o://6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32" gracePeriod=30
Sep 11 14:16:04.656817 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.656790    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:16:04.660351 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.660332    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:16:04.699846 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.699802    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btplj\" (UniqueName: \"kubernetes.io/projected/32a72900-c20d-46ea-83a5-20460dcee57b-kube-api-access-btplj\") pod \"32a72900-c20d-46ea-83a5-20460dcee57b\" (UID: \"32a72900-c20d-46ea-83a5-20460dcee57b\") "
Sep 11 14:16:04.700012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.699879    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72rtl\" (UniqueName: \"kubernetes.io/projected/3d31509c-619b-4b79-ae74-d728881d0698-kube-api-access-72rtl\") pod \"3d31509c-619b-4b79-ae74-d728881d0698\" (UID: \"3d31509c-619b-4b79-ae74-d728881d0698\") "
Sep 11 14:16:04.700012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.699964    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a72900-c20d-46ea-83a5-20460dcee57b-plugins-config-volume\") pod \"32a72900-c20d-46ea-83a5-20460dcee57b\" (UID: \"32a72900-c20d-46ea-83a5-20460dcee57b\") "
Sep 11 14:16:04.700012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.699989    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d31509c-619b-4b79-ae74-d728881d0698-plugins-config-volume\") pod \"3d31509c-619b-4b79-ae74-d728881d0698\" (UID: \"3d31509c-619b-4b79-ae74-d728881d0698\") "
Sep 11 14:16:04.700354 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.700326    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32a72900-c20d-46ea-83a5-20460dcee57b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32a72900-c20d-46ea-83a5-20460dcee57b" (UID: "32a72900-c20d-46ea-83a5-20460dcee57b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:16:04.700460 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.700422    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d31509c-619b-4b79-ae74-d728881d0698-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3d31509c-619b-4b79-ae74-d728881d0698" (UID: "3d31509c-619b-4b79-ae74-d728881d0698"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:16:04.702013 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.701993    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32a72900-c20d-46ea-83a5-20460dcee57b-kube-api-access-btplj" (OuterVolumeSpecName: "kube-api-access-btplj") pod "32a72900-c20d-46ea-83a5-20460dcee57b" (UID: "32a72900-c20d-46ea-83a5-20460dcee57b"). InnerVolumeSpecName "kube-api-access-btplj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:16:04.702099 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.702048    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d31509c-619b-4b79-ae74-d728881d0698-kube-api-access-72rtl" (OuterVolumeSpecName: "kube-api-access-72rtl") pod "3d31509c-619b-4b79-ae74-d728881d0698" (UID: "3d31509c-619b-4b79-ae74-d728881d0698"). InnerVolumeSpecName "kube-api-access-72rtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:16:04.801581 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.801547    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3d31509c-619b-4b79-ae74-d728881d0698-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:16:04.801581 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.801575    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btplj\" (UniqueName: \"kubernetes.io/projected/32a72900-c20d-46ea-83a5-20460dcee57b-kube-api-access-btplj\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:16:04.801581 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.801584    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72rtl\" (UniqueName: \"kubernetes.io/projected/3d31509c-619b-4b79-ae74-d728881d0698-kube-api-access-72rtl\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:16:04.801820 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:04.801593    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32a72900-c20d-46ea-83a5-20460dcee57b-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:16:05.222482 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.222446    2564 generic.go:356] "Generic (PLEG): container finished" podID="32a72900-c20d-46ea-83a5-20460dcee57b" containerID="6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32" exitCode=0
Sep 11 14:16:05.222696 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.222504    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"
Sep 11 14:16:05.222696 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.222541    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc" event={"ID":"32a72900-c20d-46ea-83a5-20460dcee57b","Type":"ContainerDied","Data":"6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32"}
Sep 11 14:16:05.222696 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.222607    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc" event={"ID":"32a72900-c20d-46ea-83a5-20460dcee57b","Type":"ContainerDied","Data":"53d6e3e0ab964cc7277541d4d646dc66ae2bbf45443b9a8d01e9af97c98a8404"}
Sep 11 14:16:05.222696 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.222666    2564 scope.go:117] "RemoveContainer" containerID="6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32"
Sep 11 14:16:05.224257 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.224235    2564 generic.go:356] "Generic (PLEG): container finished" podID="3d31509c-619b-4b79-ae74-d728881d0698" containerID="bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2" exitCode=0
Sep 11 14:16:05.224382 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.224268    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958" event={"ID":"3d31509c-619b-4b79-ae74-d728881d0698","Type":"ContainerDied","Data":"bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2"}
Sep 11 14:16:05.224382 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.224287    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"
Sep 11 14:16:05.224382 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.224305    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958" event={"ID":"3d31509c-619b-4b79-ae74-d728881d0698","Type":"ContainerDied","Data":"fc8bb1b28adf00ea8355d889d340a9b59144b47fe240ee153ce682fa555bef96"}
Sep 11 14:16:05.232537 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.232452    2564 scope.go:117] "RemoveContainer" containerID="6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32"
Sep 11 14:16:05.232841 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:16:05.232820    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32\": container with ID starting with 6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32 not found: ID does not exist" containerID="6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32"
Sep 11 14:16:05.232940 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.232848    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32"} err="failed to get container status \"6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32\": rpc error: code = NotFound desc = could not find container \"6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32\": container with ID starting with 6cfdedf6ea0f0672b8118f9a0ec3c9eb1e86731b29c766126d90a69ae22b0c32 not found: ID does not exist"
Sep 11 14:16:05.232940 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.232870    2564 scope.go:117] "RemoveContainer" containerID="bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2"
Sep 11 14:16:05.240693 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.240671    2564 scope.go:117] "RemoveContainer" containerID="bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2"
Sep 11 14:16:05.241030 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:16:05.241010    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2\": container with ID starting with bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2 not found: ID does not exist" containerID="bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2"
Sep 11 14:16:05.241106 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.241035    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2"} err="failed to get container status \"bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2\": rpc error: code = NotFound desc = could not find container \"bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2\": container with ID starting with bf8ad6fd2219b43197dc2123022fce980674ffacca2c547517635fbe7ada2eb2 not found: ID does not exist"
Sep 11 14:16:05.258028 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.258006    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"]
Sep 11 14:16:05.267410 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.267381    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2a21d9-97df49886-42hhc"]
Sep 11 14:16:05.295818 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.295793    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"]
Sep 11 14:16:05.308618 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:05.308595    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-2a21d9-5cf44fb47-kb958"]
Sep 11 14:16:06.275661 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:06.275603    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32a72900-c20d-46ea-83a5-20460dcee57b" path="/var/lib/kubelet/pods/32a72900-c20d-46ea-83a5-20460dcee57b/volumes"
Sep 11 14:16:06.276107 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:16:06.275990    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d31509c-619b-4b79-ae74-d728881d0698" path="/var/lib/kubelet/pods/3d31509c-619b-4b79-ae74-d728881d0698/volumes"
Sep 11 14:17:09.368109 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368077    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"]
Sep 11 14:17:09.368533 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368458    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32a72900-c20d-46ea-83a5-20460dcee57b" containerName="payload-pre-processing"
Sep 11 14:17:09.368533 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368469    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="32a72900-c20d-46ea-83a5-20460dcee57b" containerName="payload-pre-processing"
Sep 11 14:17:09.368533 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368502    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="433f09d8-cef4-4171-ae26-7bf134fe1241" containerName="maas-api"
Sep 11 14:17:09.368533 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368510    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="433f09d8-cef4-4171-ae26-7bf134fe1241" containerName="maas-api"
Sep 11 14:17:09.368533 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368518    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d31509c-619b-4b79-ae74-d728881d0698" containerName="payload-processing"
Sep 11 14:17:09.368533 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368524    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d31509c-619b-4b79-ae74-d728881d0698" containerName="payload-processing"
Sep 11 14:17:09.368760 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368592    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="433f09d8-cef4-4171-ae26-7bf134fe1241" containerName="maas-api"
Sep 11 14:17:09.368760 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368602    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="32a72900-c20d-46ea-83a5-20460dcee57b" containerName="payload-pre-processing"
Sep 11 14:17:09.368760 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.368608    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d31509c-619b-4b79-ae74-d728881d0698" containerName="payload-processing"
Sep 11 14:17:09.371833 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.371815    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.374191 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.374174    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-bef04b-serving-cert\""
Sep 11 14:17:09.389434 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.389409    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"]
Sep 11 14:17:09.479896 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.479863    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a770b4da-ecec-48fd-b007-60b01a029dd4-openshift-service-ca\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.480084 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.479978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-metrics-certs\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.480084 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.480025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpbt6\" (UniqueName: \"kubernetes.io/projected/a770b4da-ecec-48fd-b007-60b01a029dd4-kube-api-access-dpbt6\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.480084 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.480065    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-maas-api-tls\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.580818 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.580784    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a770b4da-ecec-48fd-b007-60b01a029dd4-openshift-service-ca\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.581007 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.580861    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-metrics-certs\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.581007 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.580891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpbt6\" (UniqueName: \"kubernetes.io/projected/a770b4da-ecec-48fd-b007-60b01a029dd4-kube-api-access-dpbt6\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.581007 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.580921    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-maas-api-tls\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.581458 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.581431    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a770b4da-ecec-48fd-b007-60b01a029dd4-openshift-service-ca\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.583283 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.583260    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-maas-api-tls\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.583387 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.583330    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-metrics-certs\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.600392 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.600372    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpbt6\" (UniqueName: \"kubernetes.io/projected/a770b4da-ecec-48fd-b007-60b01a029dd4-kube-api-access-dpbt6\") pod \"maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.681929 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.681859    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:09.970097 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:09.970058    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"]
Sep 11 14:17:09.972311 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:17:09.972281    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda770b4da_ecec_48fd_b007_60b01a029dd4.slice/crio-4464032c9bad4a3c1245172b04f4ee3997c245c7a92a6be214565aa6312959b2 WatchSource:0}: Error finding container 4464032c9bad4a3c1245172b04f4ee3997c245c7a92a6be214565aa6312959b2: Status 404 returned error can't find the container with id 4464032c9bad4a3c1245172b04f4ee3997c245c7a92a6be214565aa6312959b2
Sep 11 14:17:10.371452 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.371415    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"]
Sep 11 14:17:10.376156 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.376137    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:10.380269 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.380244    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-bef04b\""
Sep 11 14:17:10.380269 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.380249    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-bef04b-dockercfg-lzqxb\""
Sep 11 14:17:10.409205 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.409180    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"]
Sep 11 14:17:10.489876 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.489850    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txqsd\" (UniqueName: \"kubernetes.io/projected/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-kube-api-access-txqsd\") pod \"payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g\" (UID: \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:10.489996 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.489975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g\" (UID: \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:10.504147 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.504122    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"]
Sep 11 14:17:10.507600 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.507580    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:17:10.530020 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.529995    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8" event={"ID":"a770b4da-ecec-48fd-b007-60b01a029dd4","Type":"ContainerStarted","Data":"7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7"}
Sep 11 14:17:10.530130 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.530043    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:10.530130 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.530062    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8" event={"ID":"a770b4da-ecec-48fd-b007-60b01a029dd4","Type":"ContainerStarted","Data":"4464032c9bad4a3c1245172b04f4ee3997c245c7a92a6be214565aa6312959b2"}
Sep 11 14:17:10.552031 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.551999    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"]
Sep 11 14:17:10.590530 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.590501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g\" (UID: \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:10.590733 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.590613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txqsd\" (UniqueName: \"kubernetes.io/projected/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-kube-api-access-txqsd\") pod \"payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g\" (UID: \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:10.591317 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.591295    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g\" (UID: \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:10.609093 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.608996    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8" podStartSLOduration=1.173464465 podStartE2EDuration="1.608978265s" totalImagesPullingTime="435.5138ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:17:10.605773085 +0000 UTC m=+1180.943348645" watchObservedRunningTime="2026-09-11 14:17:10.608978265 +0000 UTC m=+1180.946553858"
Sep 11 14:17:10.611228 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.611209    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txqsd\" (UniqueName: \"kubernetes.io/projected/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-kube-api-access-txqsd\") pod \"payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g\" (UID: \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:10.687092 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.687022    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:10.691232 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.691213    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb78e267-4ad6-4abe-8039-a205124ada3e-plugins-config-volume\") pod \"payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk\" (UID: \"cb78e267-4ad6-4abe-8039-a205124ada3e\") " pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:17:10.691518 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.691494    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5ttl\" (UniqueName: \"kubernetes.io/projected/cb78e267-4ad6-4abe-8039-a205124ada3e-kube-api-access-v5ttl\") pod \"payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk\" (UID: \"cb78e267-4ad6-4abe-8039-a205124ada3e\") " pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:17:10.792321 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.792278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5ttl\" (UniqueName: \"kubernetes.io/projected/cb78e267-4ad6-4abe-8039-a205124ada3e-kube-api-access-v5ttl\") pod \"payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk\" (UID: \"cb78e267-4ad6-4abe-8039-a205124ada3e\") " pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:17:10.792505 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.792360    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb78e267-4ad6-4abe-8039-a205124ada3e-plugins-config-volume\") pod \"payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk\" (UID: \"cb78e267-4ad6-4abe-8039-a205124ada3e\") " pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:17:10.793118 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.793094    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb78e267-4ad6-4abe-8039-a205124ada3e-plugins-config-volume\") pod \"payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk\" (UID: \"cb78e267-4ad6-4abe-8039-a205124ada3e\") " pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:17:10.807103 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.807070    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5ttl\" (UniqueName: \"kubernetes.io/projected/cb78e267-4ad6-4abe-8039-a205124ada3e-kube-api-access-v5ttl\") pod \"payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk\" (UID: \"cb78e267-4ad6-4abe-8039-a205124ada3e\") " pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:17:10.817819 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.817796    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:17:10.982993 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:10.982968    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"]
Sep 11 14:17:10.984857 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:17:10.984829    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8fb937_b59a_4ab0_9a3e_09ba13ae1aaf.slice/crio-50ae988d9d01afd6b54a2099f45482276af81a22d8a80dfc18d6a350984bfd16 WatchSource:0}: Error finding container 50ae988d9d01afd6b54a2099f45482276af81a22d8a80dfc18d6a350984bfd16: Status 404 returned error can't find the container with id 50ae988d9d01afd6b54a2099f45482276af81a22d8a80dfc18d6a350984bfd16
Sep 11 14:17:11.113292 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:11.113263    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"]
Sep 11 14:17:11.113816 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:17:11.113792    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb78e267_4ad6_4abe_8039_a205124ada3e.slice/crio-dfefd4710c558d4dd34d403396e602825a3f360a198bb3c8241f3bd4fbee288f WatchSource:0}: Error finding container dfefd4710c558d4dd34d403396e602825a3f360a198bb3c8241f3bd4fbee288f: Status 404 returned error can't find the container with id dfefd4710c558d4dd34d403396e602825a3f360a198bb3c8241f3bd4fbee288f
Sep 11 14:17:11.537214 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:11.537176    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g" event={"ID":"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf","Type":"ContainerStarted","Data":"bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135"}
Sep 11 14:17:11.537728 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:11.537328    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:11.537728 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:11.537358    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g" event={"ID":"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf","Type":"ContainerStarted","Data":"50ae988d9d01afd6b54a2099f45482276af81a22d8a80dfc18d6a350984bfd16"}
Sep 11 14:17:11.538833 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:11.538810    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk" event={"ID":"cb78e267-4ad6-4abe-8039-a205124ada3e","Type":"ContainerStarted","Data":"5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072"}
Sep 11 14:17:11.538959 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:11.538859    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:17:11.538959 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:11.538882    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk" event={"ID":"cb78e267-4ad6-4abe-8039-a205124ada3e","Type":"ContainerStarted","Data":"dfefd4710c558d4dd34d403396e602825a3f360a198bb3c8241f3bd4fbee288f"}
Sep 11 14:17:11.574364 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:11.574291    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g" podStartSLOduration=1.574273597 podStartE2EDuration="1.574273597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:17:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:17:11.574197535 +0000 UTC m=+1181.911773096" watchObservedRunningTime="2026-09-11 14:17:11.574273597 +0000 UTC m=+1181.911849158"
Sep 11 14:17:11.614780 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:11.614699    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk" podStartSLOduration=1.61467807 podStartE2EDuration="1.61467807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:17:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:17:11.609702623 +0000 UTC m=+1181.947278186" watchObservedRunningTime="2026-09-11 14:17:11.61467807 +0000 UTC m=+1181.952253635"
Sep 11 14:17:16.544117 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:16.544087    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:17:22.545534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:22.545504    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:17:22.545943 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:17:22.545611    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:18:10.912290 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:10.912253    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"]
Sep 11 14:18:10.915821 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:10.915804    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:10.919019 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:10.918996    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-20588104\""
Sep 11 14:18:10.919478 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:10.919462    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-20588104-dockercfg-67w9b\""
Sep 11 14:18:10.928548 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:10.928529    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bddbb9a4-573e-4295-9694-53eee9cdc942-plugins-config-volume\") pod \"payload-processing-e2e-mt-20588104-748c69fff7-xhvp2\" (UID: \"bddbb9a4-573e-4295-9694-53eee9cdc942\") " pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:10.928646 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:10.928556    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5qjv\" (UniqueName: \"kubernetes.io/projected/bddbb9a4-573e-4295-9694-53eee9cdc942-kube-api-access-t5qjv\") pod \"payload-processing-e2e-mt-20588104-748c69fff7-xhvp2\" (UID: \"bddbb9a4-573e-4295-9694-53eee9cdc942\") " pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:10.940402 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:10.940384    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"]
Sep 11 14:18:11.029157 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.029122    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bddbb9a4-573e-4295-9694-53eee9cdc942-plugins-config-volume\") pod \"payload-processing-e2e-mt-20588104-748c69fff7-xhvp2\" (UID: \"bddbb9a4-573e-4295-9694-53eee9cdc942\") " pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:11.029341 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.029162    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5qjv\" (UniqueName: \"kubernetes.io/projected/bddbb9a4-573e-4295-9694-53eee9cdc942-kube-api-access-t5qjv\") pod \"payload-processing-e2e-mt-20588104-748c69fff7-xhvp2\" (UID: \"bddbb9a4-573e-4295-9694-53eee9cdc942\") " pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:11.029830 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.029809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bddbb9a4-573e-4295-9694-53eee9cdc942-plugins-config-volume\") pod \"payload-processing-e2e-mt-20588104-748c69fff7-xhvp2\" (UID: \"bddbb9a4-573e-4295-9694-53eee9cdc942\") " pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:11.043553 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.043532    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5qjv\" (UniqueName: \"kubernetes.io/projected/bddbb9a4-573e-4295-9694-53eee9cdc942-kube-api-access-t5qjv\") pod \"payload-processing-e2e-mt-20588104-748c69fff7-xhvp2\" (UID: \"bddbb9a4-573e-4295-9694-53eee9cdc942\") " pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:11.226126 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.226042    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:11.518620 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.518541    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"]
Sep 11 14:18:11.521991 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:18:11.521968    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbddbb9a4_573e_4295_9694_53eee9cdc942.slice/crio-ac7a21a8693da5fe7779e2b01d2d3685d865fb6423be4d9416f7f465766b719e WatchSource:0}: Error finding container ac7a21a8693da5fe7779e2b01d2d3685d865fb6423be4d9416f7f465766b719e: Status 404 returned error can't find the container with id ac7a21a8693da5fe7779e2b01d2d3685d865fb6423be4d9416f7f465766b719e
Sep 11 14:18:11.838553 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.838514    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2" event={"ID":"bddbb9a4-573e-4295-9694-53eee9cdc942","Type":"ContainerStarted","Data":"0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462"}
Sep 11 14:18:11.838748 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.838575    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2" event={"ID":"bddbb9a4-573e-4295-9694-53eee9cdc942","Type":"ContainerStarted","Data":"ac7a21a8693da5fe7779e2b01d2d3685d865fb6423be4d9416f7f465766b719e"}
Sep 11 14:18:11.838748 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.838663    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:11.862506 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:11.862461    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2" podStartSLOduration=1.862445359 podStartE2EDuration="1.862445359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:18:11.860016767 +0000 UTC m=+1242.197592326" watchObservedRunningTime="2026-09-11 14:18:11.862445359 +0000 UTC m=+1242.200020918"
Sep 11 14:18:22.846800 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:22.846718    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:26.013514 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.013476    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"]
Sep 11 14:18:26.013905 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.013742    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8" podUID="a770b4da-ecec-48fd-b007-60b01a029dd4" containerName="maas-api" containerID="cri-o://7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7" gracePeriod=30
Sep 11 14:18:26.123023 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.122990    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"]
Sep 11 14:18:26.123247 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.123210    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk" podUID="cb78e267-4ad6-4abe-8039-a205124ada3e" containerName="payload-processing" containerID="cri-o://5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072" gracePeriod=30
Sep 11 14:18:26.143779 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.143752    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"]
Sep 11 14:18:26.144003 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.143960    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g" podUID="cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf" containerName="payload-pre-processing" containerID="cri-o://bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135" gracePeriod=30
Sep 11 14:18:26.268774 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.268704    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:18:26.375270 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.375235    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-metrics-certs\") pod \"a770b4da-ecec-48fd-b007-60b01a029dd4\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") "
Sep 11 14:18:26.375451 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.375304    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-maas-api-tls\") pod \"a770b4da-ecec-48fd-b007-60b01a029dd4\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") "
Sep 11 14:18:26.375451 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.375386    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a770b4da-ecec-48fd-b007-60b01a029dd4-openshift-service-ca\") pod \"a770b4da-ecec-48fd-b007-60b01a029dd4\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") "
Sep 11 14:18:26.375451 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.375446    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpbt6\" (UniqueName: \"kubernetes.io/projected/a770b4da-ecec-48fd-b007-60b01a029dd4-kube-api-access-dpbt6\") pod \"a770b4da-ecec-48fd-b007-60b01a029dd4\" (UID: \"a770b4da-ecec-48fd-b007-60b01a029dd4\") "
Sep 11 14:18:26.375809 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.375774    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a770b4da-ecec-48fd-b007-60b01a029dd4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a770b4da-ecec-48fd-b007-60b01a029dd4" (UID: "a770b4da-ecec-48fd-b007-60b01a029dd4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:18:26.378513 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.378480    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a770b4da-ecec-48fd-b007-60b01a029dd4" (UID: "a770b4da-ecec-48fd-b007-60b01a029dd4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:18:26.378513 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.378497    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a770b4da-ecec-48fd-b007-60b01a029dd4-kube-api-access-dpbt6" (OuterVolumeSpecName: "kube-api-access-dpbt6") pod "a770b4da-ecec-48fd-b007-60b01a029dd4" (UID: "a770b4da-ecec-48fd-b007-60b01a029dd4"). InnerVolumeSpecName "kube-api-access-dpbt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:18:26.378669 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.378580    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a770b4da-ecec-48fd-b007-60b01a029dd4" (UID: "a770b4da-ecec-48fd-b007-60b01a029dd4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:18:26.396526 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.396504    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:18:26.405326 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.405306    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:18:26.476819 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.476784    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5ttl\" (UniqueName: \"kubernetes.io/projected/cb78e267-4ad6-4abe-8039-a205124ada3e-kube-api-access-v5ttl\") pod \"cb78e267-4ad6-4abe-8039-a205124ada3e\" (UID: \"cb78e267-4ad6-4abe-8039-a205124ada3e\") "
Sep 11 14:18:26.477008 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.476857    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-plugins-config-volume\") pod \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\" (UID: \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\") "
Sep 11 14:18:26.477008 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.476886    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txqsd\" (UniqueName: \"kubernetes.io/projected/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-kube-api-access-txqsd\") pod \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\" (UID: \"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf\") "
Sep 11 14:18:26.477008 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.476954    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb78e267-4ad6-4abe-8039-a205124ada3e-plugins-config-volume\") pod \"cb78e267-4ad6-4abe-8039-a205124ada3e\" (UID: \"cb78e267-4ad6-4abe-8039-a205124ada3e\") "
Sep 11 14:18:26.477287 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.477235    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:26.477287 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.477255    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a770b4da-ecec-48fd-b007-60b01a029dd4-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:26.477287 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.477269    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a770b4da-ecec-48fd-b007-60b01a029dd4-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:26.477287 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.477263    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf" (UID: "cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:18:26.477287 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.477285    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpbt6\" (UniqueName: \"kubernetes.io/projected/a770b4da-ecec-48fd-b007-60b01a029dd4-kube-api-access-dpbt6\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:26.477643 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.477598    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb78e267-4ad6-4abe-8039-a205124ada3e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cb78e267-4ad6-4abe-8039-a205124ada3e" (UID: "cb78e267-4ad6-4abe-8039-a205124ada3e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:18:26.478946 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.478928    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-kube-api-access-txqsd" (OuterVolumeSpecName: "kube-api-access-txqsd") pod "cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf" (UID: "cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf"). InnerVolumeSpecName "kube-api-access-txqsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:18:26.479409 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.479389    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb78e267-4ad6-4abe-8039-a205124ada3e-kube-api-access-v5ttl" (OuterVolumeSpecName: "kube-api-access-v5ttl") pod "cb78e267-4ad6-4abe-8039-a205124ada3e" (UID: "cb78e267-4ad6-4abe-8039-a205124ada3e"). InnerVolumeSpecName "kube-api-access-v5ttl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:18:26.578644 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.578610    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5ttl\" (UniqueName: \"kubernetes.io/projected/cb78e267-4ad6-4abe-8039-a205124ada3e-kube-api-access-v5ttl\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:26.578644 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.578649    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:26.578835 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.578663    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txqsd\" (UniqueName: \"kubernetes.io/projected/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf-kube-api-access-txqsd\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:26.578835 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.578675    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb78e267-4ad6-4abe-8039-a205124ada3e-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:26.915568 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.915485    2564 generic.go:356] "Generic (PLEG): container finished" podID="cb78e267-4ad6-4abe-8039-a205124ada3e" containerID="5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072" exitCode=0
Sep 11 14:18:26.915568 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.915525    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"
Sep 11 14:18:26.916036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.915576    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk" event={"ID":"cb78e267-4ad6-4abe-8039-a205124ada3e","Type":"ContainerDied","Data":"5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072"}
Sep 11 14:18:26.916036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.915657    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk" event={"ID":"cb78e267-4ad6-4abe-8039-a205124ada3e","Type":"ContainerDied","Data":"dfefd4710c558d4dd34d403396e602825a3f360a198bb3c8241f3bd4fbee288f"}
Sep 11 14:18:26.916036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.915689    2564 scope.go:117] "RemoveContainer" containerID="5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072"
Sep 11 14:18:26.917005 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.916979    2564 generic.go:356] "Generic (PLEG): container finished" podID="a770b4da-ecec-48fd-b007-60b01a029dd4" containerID="7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7" exitCode=0
Sep 11 14:18:26.917114 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.917010    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8" event={"ID":"a770b4da-ecec-48fd-b007-60b01a029dd4","Type":"ContainerDied","Data":"7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7"}
Sep 11 14:18:26.917114 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.917034    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"
Sep 11 14:18:26.917114 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.917056    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8" event={"ID":"a770b4da-ecec-48fd-b007-60b01a029dd4","Type":"ContainerDied","Data":"4464032c9bad4a3c1245172b04f4ee3997c245c7a92a6be214565aa6312959b2"}
Sep 11 14:18:26.918755 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.918730    2564 generic.go:356] "Generic (PLEG): container finished" podID="cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf" containerID="bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135" exitCode=0
Sep 11 14:18:26.918856 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.918774    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"
Sep 11 14:18:26.918856 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.918779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g" event={"ID":"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf","Type":"ContainerDied","Data":"bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135"}
Sep 11 14:18:26.918856 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.918815    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g" event={"ID":"cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf","Type":"ContainerDied","Data":"50ae988d9d01afd6b54a2099f45482276af81a22d8a80dfc18d6a350984bfd16"}
Sep 11 14:18:26.925507 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.925488    2564 scope.go:117] "RemoveContainer" containerID="5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072"
Sep 11 14:18:26.925819 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:18:26.925797    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072\": container with ID starting with 5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072 not found: ID does not exist" containerID="5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072"
Sep 11 14:18:26.925927 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.925825    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072"} err="failed to get container status \"5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072\": rpc error: code = NotFound desc = could not find container \"5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072\": container with ID starting with 5fd9497575ec07c0c60c6d360e431378c2139bdc6aae11e4366f17ba82df0072 not found: ID does not exist"
Sep 11 14:18:26.925927 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.925841    2564 scope.go:117] "RemoveContainer" containerID="7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7"
Sep 11 14:18:26.934617 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.934602    2564 scope.go:117] "RemoveContainer" containerID="7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7"
Sep 11 14:18:26.934933 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:18:26.934913    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7\": container with ID starting with 7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7 not found: ID does not exist" containerID="7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7"
Sep 11 14:18:26.934988 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.934939    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7"} err="failed to get container status \"7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7\": rpc error: code = NotFound desc = could not find container \"7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7\": container with ID starting with 7b1c3ba9e749d59bfc6327304d06d76e04b40ab0cac0308e63b60cc802791be7 not found: ID does not exist"
Sep 11 14:18:26.934988 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.934956    2564 scope.go:117] "RemoveContainer" containerID="bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135"
Sep 11 14:18:26.942294 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.942277    2564 scope.go:117] "RemoveContainer" containerID="bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135"
Sep 11 14:18:26.942558 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:18:26.942537    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135\": container with ID starting with bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135 not found: ID does not exist" containerID="bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135"
Sep 11 14:18:26.942661 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.942560    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135"} err="failed to get container status \"bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135\": rpc error: code = NotFound desc = could not find container \"bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135\": container with ID starting with bfffe0acc87749410218ca8d3fae93b2107899b56726747936fbbe8a49bcc135 not found: ID does not exist"
Sep 11 14:18:26.956002 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.955980    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"]
Sep 11 14:18:26.961308 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.961286    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-bef04b-7d6c9658bf-gthzk"]
Sep 11 14:18:26.994137 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:26.994111    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"]
Sep 11 14:18:27.013933 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:27.013910    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-bef04b-6ff6b45769-k5hz8"]
Sep 11 14:18:27.033979 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:27.033956    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"]
Sep 11 14:18:27.037939 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:27.037916    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-bef04b-7d5644c4b6-wdc5g"]
Sep 11 14:18:28.276232 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:28.276201    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a770b4da-ecec-48fd-b007-60b01a029dd4" path="/var/lib/kubelet/pods/a770b4da-ecec-48fd-b007-60b01a029dd4/volumes"
Sep 11 14:18:28.276605 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:28.276552    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb78e267-4ad6-4abe-8039-a205124ada3e" path="/var/lib/kubelet/pods/cb78e267-4ad6-4abe-8039-a205124ada3e/volumes"
Sep 11 14:18:28.276910 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:28.276897    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf" path="/var/lib/kubelet/pods/cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf/volumes"
Sep 11 14:18:57.288858 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:57.288824    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"]
Sep 11 14:18:57.289358 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:57.289069    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2" podUID="bddbb9a4-573e-4295-9694-53eee9cdc942" containerName="payload-processing" containerID="cri-o://0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462" gracePeriod=30
Sep 11 14:18:57.545651 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:57.545580    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:57.649083 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:57.649047    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bddbb9a4-573e-4295-9694-53eee9cdc942-plugins-config-volume\") pod \"bddbb9a4-573e-4295-9694-53eee9cdc942\" (UID: \"bddbb9a4-573e-4295-9694-53eee9cdc942\") "
Sep 11 14:18:57.649274 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:57.649089    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5qjv\" (UniqueName: \"kubernetes.io/projected/bddbb9a4-573e-4295-9694-53eee9cdc942-kube-api-access-t5qjv\") pod \"bddbb9a4-573e-4295-9694-53eee9cdc942\" (UID: \"bddbb9a4-573e-4295-9694-53eee9cdc942\") "
Sep 11 14:18:57.649520 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:57.649497    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bddbb9a4-573e-4295-9694-53eee9cdc942-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bddbb9a4-573e-4295-9694-53eee9cdc942" (UID: "bddbb9a4-573e-4295-9694-53eee9cdc942"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:18:57.651464 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:57.651443    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bddbb9a4-573e-4295-9694-53eee9cdc942-kube-api-access-t5qjv" (OuterVolumeSpecName: "kube-api-access-t5qjv") pod "bddbb9a4-573e-4295-9694-53eee9cdc942" (UID: "bddbb9a4-573e-4295-9694-53eee9cdc942"). InnerVolumeSpecName "kube-api-access-t5qjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:18:57.750091 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:57.750058    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bddbb9a4-573e-4295-9694-53eee9cdc942-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:57.750091 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:57.750086    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5qjv\" (UniqueName: \"kubernetes.io/projected/bddbb9a4-573e-4295-9694-53eee9cdc942-kube-api-access-t5qjv\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:18:58.079389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.079358    2564 generic.go:356] "Generic (PLEG): container finished" podID="bddbb9a4-573e-4295-9694-53eee9cdc942" containerID="0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462" exitCode=0
Sep 11 14:18:58.079569 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.079404    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"
Sep 11 14:18:58.079569 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.079415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2" event={"ID":"bddbb9a4-573e-4295-9694-53eee9cdc942","Type":"ContainerDied","Data":"0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462"}
Sep 11 14:18:58.079569 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.079454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2" event={"ID":"bddbb9a4-573e-4295-9694-53eee9cdc942","Type":"ContainerDied","Data":"ac7a21a8693da5fe7779e2b01d2d3685d865fb6423be4d9416f7f465766b719e"}
Sep 11 14:18:58.079569 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.079474    2564 scope.go:117] "RemoveContainer" containerID="0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462"
Sep 11 14:18:58.088476 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.088457    2564 scope.go:117] "RemoveContainer" containerID="0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462"
Sep 11 14:18:58.088800 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:18:58.088778    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462\": container with ID starting with 0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462 not found: ID does not exist" containerID="0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462"
Sep 11 14:18:58.088944 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.088809    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462"} err="failed to get container status \"0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462\": rpc error: code = NotFound desc = could not find container \"0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462\": container with ID starting with 0ea1bddca0284f9347abea631ac8199b09235eb42c6fad71ed15e424b58ac462 not found: ID does not exist"
Sep 11 14:18:58.114588 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.114562    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"]
Sep 11 14:18:58.121330 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.121308    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-20588104-748c69fff7-xhvp2"]
Sep 11 14:18:58.274900 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:18:58.274870    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bddbb9a4-573e-4295-9694-53eee9cdc942" path="/var/lib/kubelet/pods/bddbb9a4-573e-4295-9694-53eee9cdc942/volumes"
Sep 11 14:19:18.629160 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:18.629126    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"]
Sep 11 14:19:18.629561 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:18.629337    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb" podUID="204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" containerName="maas-api" containerID="cri-o://83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e" gracePeriod=30
Sep 11 14:19:18.731308 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:18.731276    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"]
Sep 11 14:19:18.731518 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:18.731479    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4" podUID="e449eacc-b976-40b7-9c53-d5bc817f18d2" containerName="payload-processing" containerID="cri-o://ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f" gracePeriod=30
Sep 11 14:19:18.754759 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:18.754730    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"]
Sep 11 14:19:18.755006 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:18.754982    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk" podUID="6108161b-a6ee-4ba0-a061-277e90658ea2" containerName="payload-pre-processing" containerID="cri-o://dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e" gracePeriod=30
Sep 11 14:19:18.890702 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:18.890621    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:19:18.993242 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:18.993208    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:19:19.040822 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.040795    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:19:19.048939 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.048915    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sngk7\" (UniqueName: \"kubernetes.io/projected/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-kube-api-access-sngk7\") pod \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") "
Sep 11 14:19:19.049049 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.048970    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-maas-api-tls\") pod \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") "
Sep 11 14:19:19.049049 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.048997    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-metrics-certs\") pod \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") "
Sep 11 14:19:19.049181 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.049122    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-openshift-service-ca\") pod \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\" (UID: \"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0\") "
Sep 11 14:19:19.049479 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.049456    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" (UID: "204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:19:19.050980 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.050956    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" (UID: "204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:19:19.051064 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.051035    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" (UID: "204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:19:19.051120 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.051073    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-kube-api-access-sngk7" (OuterVolumeSpecName: "kube-api-access-sngk7") pod "204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" (UID: "204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0"). InnerVolumeSpecName "kube-api-access-sngk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:19:19.150379 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150300    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsk4x\" (UniqueName: \"kubernetes.io/projected/6108161b-a6ee-4ba0-a061-277e90658ea2-kube-api-access-bsk4x\") pod \"6108161b-a6ee-4ba0-a061-277e90658ea2\" (UID: \"6108161b-a6ee-4ba0-a061-277e90658ea2\") "
Sep 11 14:19:19.150379 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150352    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6108161b-a6ee-4ba0-a061-277e90658ea2-plugins-config-volume\") pod \"6108161b-a6ee-4ba0-a061-277e90658ea2\" (UID: \"6108161b-a6ee-4ba0-a061-277e90658ea2\") "
Sep 11 14:19:19.150379 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150373    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e449eacc-b976-40b7-9c53-d5bc817f18d2-plugins-config-volume\") pod \"e449eacc-b976-40b7-9c53-d5bc817f18d2\" (UID: \"e449eacc-b976-40b7-9c53-d5bc817f18d2\") "
Sep 11 14:19:19.150597 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150426    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9r9cd\" (UniqueName: \"kubernetes.io/projected/e449eacc-b976-40b7-9c53-d5bc817f18d2-kube-api-access-9r9cd\") pod \"e449eacc-b976-40b7-9c53-d5bc817f18d2\" (UID: \"e449eacc-b976-40b7-9c53-d5bc817f18d2\") "
Sep 11 14:19:19.150736 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150717    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sngk7\" (UniqueName: \"kubernetes.io/projected/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-kube-api-access-sngk7\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:19.150816 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150742    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:19.150816 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150756    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:19.150816 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150770    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:19.150956 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150814    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6108161b-a6ee-4ba0-a061-277e90658ea2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6108161b-a6ee-4ba0-a061-277e90658ea2" (UID: "6108161b-a6ee-4ba0-a061-277e90658ea2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:19:19.150956 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.150821    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e449eacc-b976-40b7-9c53-d5bc817f18d2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e449eacc-b976-40b7-9c53-d5bc817f18d2" (UID: "e449eacc-b976-40b7-9c53-d5bc817f18d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:19:19.152540 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.152515    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e449eacc-b976-40b7-9c53-d5bc817f18d2-kube-api-access-9r9cd" (OuterVolumeSpecName: "kube-api-access-9r9cd") pod "e449eacc-b976-40b7-9c53-d5bc817f18d2" (UID: "e449eacc-b976-40b7-9c53-d5bc817f18d2"). InnerVolumeSpecName "kube-api-access-9r9cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:19:19.152540 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.152529    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6108161b-a6ee-4ba0-a061-277e90658ea2-kube-api-access-bsk4x" (OuterVolumeSpecName: "kube-api-access-bsk4x") pod "6108161b-a6ee-4ba0-a061-277e90658ea2" (UID: "6108161b-a6ee-4ba0-a061-277e90658ea2"). InnerVolumeSpecName "kube-api-access-bsk4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:19:19.193656 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.193596    2564 generic.go:356] "Generic (PLEG): container finished" podID="204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" containerID="83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e" exitCode=0
Sep 11 14:19:19.193834 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.193688    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"
Sep 11 14:19:19.193834 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.193682    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb" event={"ID":"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0","Type":"ContainerDied","Data":"83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e"}
Sep 11 14:19:19.193937 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.193847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb" event={"ID":"204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0","Type":"ContainerDied","Data":"8c7678f7f64efec2912e8787f15f00ae200e1c1a4d85daa23ce8756f14f2a4fe"}
Sep 11 14:19:19.193937 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.193884    2564 scope.go:117] "RemoveContainer" containerID="83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e"
Sep 11 14:19:19.195173 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.195153    2564 generic.go:356] "Generic (PLEG): container finished" podID="6108161b-a6ee-4ba0-a061-277e90658ea2" containerID="dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e" exitCode=0
Sep 11 14:19:19.195278 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.195203    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"
Sep 11 14:19:19.195278 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.195231    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk" event={"ID":"6108161b-a6ee-4ba0-a061-277e90658ea2","Type":"ContainerDied","Data":"dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e"}
Sep 11 14:19:19.195385 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.195290    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk" event={"ID":"6108161b-a6ee-4ba0-a061-277e90658ea2","Type":"ContainerDied","Data":"ca0c5328f3e59638c2dfaca89becf9d88e10d85c4f8d92e2162df833c335a172"}
Sep 11 14:19:19.196777 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.196753    2564 generic.go:356] "Generic (PLEG): container finished" podID="e449eacc-b976-40b7-9c53-d5bc817f18d2" containerID="ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f" exitCode=0
Sep 11 14:19:19.196937 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.196783    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"
Sep 11 14:19:19.196937 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.196799    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4" event={"ID":"e449eacc-b976-40b7-9c53-d5bc817f18d2","Type":"ContainerDied","Data":"ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f"}
Sep 11 14:19:19.196937 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.196837    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4" event={"ID":"e449eacc-b976-40b7-9c53-d5bc817f18d2","Type":"ContainerDied","Data":"63370817319ff891326a144e6f297d0253858468e145ab86b4af63d1a8c9a489"}
Sep 11 14:19:19.203994 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.203979    2564 scope.go:117] "RemoveContainer" containerID="83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e"
Sep 11 14:19:19.204316 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:19:19.204298    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e\": container with ID starting with 83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e not found: ID does not exist" containerID="83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e"
Sep 11 14:19:19.204407 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.204327    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e"} err="failed to get container status \"83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e\": rpc error: code = NotFound desc = could not find container \"83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e\": container with ID starting with 83dd52da20d43966f32411a4fc773b679b9cbea021134d8bc1abbbf6018d994e not found: ID does not exist"
Sep 11 14:19:19.204407 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.204349    2564 scope.go:117] "RemoveContainer" containerID="dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e"
Sep 11 14:19:19.214855 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.214832    2564 scope.go:117] "RemoveContainer" containerID="dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e"
Sep 11 14:19:19.215139 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:19:19.215119    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e\": container with ID starting with dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e not found: ID does not exist" containerID="dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e"
Sep 11 14:19:19.215193 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.215146    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e"} err="failed to get container status \"dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e\": rpc error: code = NotFound desc = could not find container \"dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e\": container with ID starting with dd6b7cab299e663c0b7c63f054068df85a21515af558a31c1884d38e094cb17e not found: ID does not exist"
Sep 11 14:19:19.215193 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.215162    2564 scope.go:117] "RemoveContainer" containerID="ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f"
Sep 11 14:19:19.223264 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.223244    2564 scope.go:117] "RemoveContainer" containerID="ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f"
Sep 11 14:19:19.223610 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:19:19.223557    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f\": container with ID starting with ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f not found: ID does not exist" containerID="ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f"
Sep 11 14:19:19.223610 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.223596    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f"} err="failed to get container status \"ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f\": rpc error: code = NotFound desc = could not find container \"ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f\": container with ID starting with ed1bc90e15bcdc0c7ca05c96deca33eacd752b1561bd65ef234fa1c0ef9d903f not found: ID does not exist"
Sep 11 14:19:19.224992 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.224973    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"]
Sep 11 14:19:19.231605 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.231585    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-be0c80-7b5c85b785-hf8lb"]
Sep 11 14:19:19.248358 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.248336    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"]
Sep 11 14:19:19.251723 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.251700    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsk4x\" (UniqueName: \"kubernetes.io/projected/6108161b-a6ee-4ba0-a061-277e90658ea2-kube-api-access-bsk4x\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:19.251723 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.251722    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6108161b-a6ee-4ba0-a061-277e90658ea2-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:19.251893 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.251732    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e449eacc-b976-40b7-9c53-d5bc817f18d2-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:19.251893 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.251741    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r9cd\" (UniqueName: \"kubernetes.io/projected/e449eacc-b976-40b7-9c53-d5bc817f18d2-kube-api-access-9r9cd\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:19.256131 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.256106    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-be0c80-86796fb4d5-xr2p4"]
Sep 11 14:19:19.271416 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.271389    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"]
Sep 11 14:19:19.281770 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:19.281744    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-be0c80-6dbb6f9f9d-kpgzk"]
Sep 11 14:19:20.275154 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:20.275114    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" path="/var/lib/kubelet/pods/204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0/volumes"
Sep 11 14:19:20.275677 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:20.275656    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6108161b-a6ee-4ba0-a061-277e90658ea2" path="/var/lib/kubelet/pods/6108161b-a6ee-4ba0-a061-277e90658ea2/volumes"
Sep 11 14:19:20.276056 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:20.276037    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e449eacc-b976-40b7-9c53-d5bc817f18d2" path="/var/lib/kubelet/pods/e449eacc-b976-40b7-9c53-d5bc817f18d2/volumes"
Sep 11 14:19:35.044988 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.044956    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"]
Sep 11 14:19:35.045509 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045484    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" containerName="maas-api"
Sep 11 14:19:35.045509 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045508    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" containerName="maas-api"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045522    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bddbb9a4-573e-4295-9694-53eee9cdc942" containerName="payload-processing"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045532    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bddbb9a4-573e-4295-9694-53eee9cdc942" containerName="payload-processing"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045553    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a770b4da-ecec-48fd-b007-60b01a029dd4" containerName="maas-api"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045562    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a770b4da-ecec-48fd-b007-60b01a029dd4" containerName="maas-api"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045578    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf" containerName="payload-pre-processing"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045587    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf" containerName="payload-pre-processing"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045606    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e449eacc-b976-40b7-9c53-d5bc817f18d2" containerName="payload-processing"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045614    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e449eacc-b976-40b7-9c53-d5bc817f18d2" containerName="payload-processing"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045641    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6108161b-a6ee-4ba0-a061-277e90658ea2" containerName="payload-pre-processing"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045650    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6108161b-a6ee-4ba0-a061-277e90658ea2" containerName="payload-pre-processing"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045658    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb78e267-4ad6-4abe-8039-a205124ada3e" containerName="payload-processing"
Sep 11 14:19:35.045715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045666    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb78e267-4ad6-4abe-8039-a205124ada3e" containerName="payload-processing"
Sep 11 14:19:35.046293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045767    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a770b4da-ecec-48fd-b007-60b01a029dd4" containerName="maas-api"
Sep 11 14:19:35.046293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045779    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6108161b-a6ee-4ba0-a061-277e90658ea2" containerName="payload-pre-processing"
Sep 11 14:19:35.046293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045789    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd8fb937-b59a-4ab0-9a3e-09ba13ae1aaf" containerName="payload-pre-processing"
Sep 11 14:19:35.046293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045798    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="204c6ce5-5526-4ffc-a3b2-aee8c5a6d2b0" containerName="maas-api"
Sep 11 14:19:35.046293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045808    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bddbb9a4-573e-4295-9694-53eee9cdc942" containerName="payload-processing"
Sep 11 14:19:35.046293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045818    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e449eacc-b976-40b7-9c53-d5bc817f18d2" containerName="payload-processing"
Sep 11 14:19:35.046293 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.045827    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb78e267-4ad6-4abe-8039-a205124ada3e" containerName="payload-processing"
Sep 11 14:19:35.050424 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.050405    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.053451 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.053433    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-fb993c-serving-cert\""
Sep 11 14:19:35.077451 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.077428    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"]
Sep 11 14:19:35.089470 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.089449    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.089568 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.089503    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdx8k\" (UniqueName: \"kubernetes.io/projected/b4e68cc5-2e69-435c-abd6-4f36652d9b19-kube-api-access-fdx8k\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.089623 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.089577    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e68cc5-2e69-435c-abd6-4f36652d9b19-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.089623 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.089609    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.190878 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.190846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.191048 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.190895    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdx8k\" (UniqueName: \"kubernetes.io/projected/b4e68cc5-2e69-435c-abd6-4f36652d9b19-kube-api-access-fdx8k\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.191048 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.190923    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e68cc5-2e69-435c-abd6-4f36652d9b19-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.191048 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.190949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.191578 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.191542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e68cc5-2e69-435c-abd6-4f36652d9b19-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.193407 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.193381    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.193488 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.193378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.201878 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.201857    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdx8k\" (UniqueName: \"kubernetes.io/projected/b4e68cc5-2e69-435c-abd6-4f36652d9b19-kube-api-access-fdx8k\") pod \"maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.360143 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.360117    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:35.637417 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:35.637391    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"]
Sep 11 14:19:35.640167 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:19:35.640112    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e68cc5_2e69_435c_abd6_4f36652d9b19.slice/crio-8a3d7f3e75003aa07b947b3772b7c1c133db244a1ef339aec3bda908fe329a6d WatchSource:0}: Error finding container 8a3d7f3e75003aa07b947b3772b7c1c133db244a1ef339aec3bda908fe329a6d: Status 404 returned error can't find the container with id 8a3d7f3e75003aa07b947b3772b7c1c133db244a1ef339aec3bda908fe329a6d
Sep 11 14:19:36.005552 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.005470    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"]
Sep 11 14:19:36.009938 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.009922    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:36.013055 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.013033    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-fb993c-dockercfg-w6smg\""
Sep 11 14:19:36.022614 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.022595    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-fb993c\""
Sep 11 14:19:36.036341 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.036306    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"]
Sep 11 14:19:36.097646 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.097597    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwn2v\" (UniqueName: \"kubernetes.io/projected/8734bbaa-bc41-4721-8e5b-ee73aad405f0-kube-api-access-mwn2v\") pod \"payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g\" (UID: \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:36.098093 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.097709    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8734bbaa-bc41-4721-8e5b-ee73aad405f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g\" (UID: \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:36.198726 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.198691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8734bbaa-bc41-4721-8e5b-ee73aad405f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g\" (UID: \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:36.198920 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.198776    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwn2v\" (UniqueName: \"kubernetes.io/projected/8734bbaa-bc41-4721-8e5b-ee73aad405f0-kube-api-access-mwn2v\") pod \"payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g\" (UID: \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:36.199353 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.199332    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8734bbaa-bc41-4721-8e5b-ee73aad405f0-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g\" (UID: \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:36.244518 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.244497    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwn2v\" (UniqueName: \"kubernetes.io/projected/8734bbaa-bc41-4721-8e5b-ee73aad405f0-kube-api-access-mwn2v\") pod \"payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g\" (UID: \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:36.281187 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.281157    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh" event={"ID":"b4e68cc5-2e69-435c-abd6-4f36652d9b19","Type":"ContainerStarted","Data":"8a3d7f3e75003aa07b947b3772b7c1c133db244a1ef339aec3bda908fe329a6d"}
Sep 11 14:19:36.320758 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.320734    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:36.602607 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:36.602587    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"]
Sep 11 14:19:36.604957 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:19:36.604930    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8734bbaa_bc41_4721_8e5b_ee73aad405f0.slice/crio-7b434e2dbf61a9938f470e27eb08e8695a0cf663b55884b5946c815cd863d32d WatchSource:0}: Error finding container 7b434e2dbf61a9938f470e27eb08e8695a0cf663b55884b5946c815cd863d32d: Status 404 returned error can't find the container with id 7b434e2dbf61a9938f470e27eb08e8695a0cf663b55884b5946c815cd863d32d
Sep 11 14:19:37.289706 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.289674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh" event={"ID":"b4e68cc5-2e69-435c-abd6-4f36652d9b19","Type":"ContainerStarted","Data":"c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9"}
Sep 11 14:19:37.290167 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.289763    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:37.291648 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.291597    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g" event={"ID":"8734bbaa-bc41-4721-8e5b-ee73aad405f0","Type":"ContainerStarted","Data":"b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa"}
Sep 11 14:19:37.291766 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.291674    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g" event={"ID":"8734bbaa-bc41-4721-8e5b-ee73aad405f0","Type":"ContainerStarted","Data":"7b434e2dbf61a9938f470e27eb08e8695a0cf663b55884b5946c815cd863d32d"}
Sep 11 14:19:37.291766 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.291703    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:37.322888 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.322831    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh" podStartSLOduration=2.745444499 podStartE2EDuration="3.322814395s" totalImagesPullingTime="577.369896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:19:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:19:37.322616472 +0000 UTC m=+1327.660192033" watchObservedRunningTime="2026-09-11 14:19:37.322814395 +0000 UTC m=+1327.660389956"
Sep 11 14:19:37.406214 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.406163    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g" podStartSLOduration=2.406146273 podStartE2EDuration="2.406146273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:19:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:19:37.355122506 +0000 UTC m=+1327.692698073" watchObservedRunningTime="2026-09-11 14:19:37.406146273 +0000 UTC m=+1327.743721832"
Sep 11 14:19:37.408079 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.408055    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth"]
Sep 11 14:19:37.415700 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.415674    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c6156e-29818935-98nth"]
Sep 11 14:19:37.470577 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.470544    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"]
Sep 11 14:19:37.470789 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.470762    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z" podUID="3cfca453-a275-465b-94fa-facb441835b1" containerName="payload-processing" containerID="cri-o://4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62" gracePeriod=30
Sep 11 14:19:37.733063 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.733039    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:19:37.813378 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.813351    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfca453-a275-465b-94fa-facb441835b1-plugins-config-volume\") pod \"3cfca453-a275-465b-94fa-facb441835b1\" (UID: \"3cfca453-a275-465b-94fa-facb441835b1\") "
Sep 11 14:19:37.813515 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.813412    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfqvf\" (UniqueName: \"kubernetes.io/projected/3cfca453-a275-465b-94fa-facb441835b1-kube-api-access-wfqvf\") pod \"3cfca453-a275-465b-94fa-facb441835b1\" (UID: \"3cfca453-a275-465b-94fa-facb441835b1\") "
Sep 11 14:19:37.813788 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.813766    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cfca453-a275-465b-94fa-facb441835b1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3cfca453-a275-465b-94fa-facb441835b1" (UID: "3cfca453-a275-465b-94fa-facb441835b1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:19:37.815497 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.815475    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cfca453-a275-465b-94fa-facb441835b1-kube-api-access-wfqvf" (OuterVolumeSpecName: "kube-api-access-wfqvf") pod "3cfca453-a275-465b-94fa-facb441835b1" (UID: "3cfca453-a275-465b-94fa-facb441835b1"). InnerVolumeSpecName "kube-api-access-wfqvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:19:37.914381 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.914303    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3cfca453-a275-465b-94fa-facb441835b1-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:37.914381 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:37.914331    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfqvf\" (UniqueName: \"kubernetes.io/projected/3cfca453-a275-465b-94fa-facb441835b1-kube-api-access-wfqvf\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:19:38.276222 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.276146    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7007cff9-b62d-4ac1-9950-bf9b9123455c" path="/var/lib/kubelet/pods/7007cff9-b62d-4ac1-9950-bf9b9123455c/volumes"
Sep 11 14:19:38.297796 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.297765    2564 generic.go:356] "Generic (PLEG): container finished" podID="3cfca453-a275-465b-94fa-facb441835b1" containerID="4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62" exitCode=0
Sep 11 14:19:38.298195 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.297804    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"
Sep 11 14:19:38.298195 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.297845    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z" event={"ID":"3cfca453-a275-465b-94fa-facb441835b1","Type":"ContainerDied","Data":"4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62"}
Sep 11 14:19:38.298195 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.297897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z" event={"ID":"3cfca453-a275-465b-94fa-facb441835b1","Type":"ContainerDied","Data":"c8bba5c19ad6203ae860901fd9ac819637f32fbd7d222cbaed3f383edad0936a"}
Sep 11 14:19:38.298195 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.297926    2564 scope.go:117] "RemoveContainer" containerID="4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62"
Sep 11 14:19:38.306850 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.306833    2564 scope.go:117] "RemoveContainer" containerID="4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62"
Sep 11 14:19:38.307120 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:19:38.307100    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62\": container with ID starting with 4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62 not found: ID does not exist" containerID="4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62"
Sep 11 14:19:38.307186 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.307130    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62"} err="failed to get container status \"4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62\": rpc error: code = NotFound desc = could not find container \"4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62\": container with ID starting with 4af4574d7a0e55111bc9bcce13de766e301a70b2259d31a8d64287ebe4dccb62 not found: ID does not exist"
Sep 11 14:19:38.324273 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.324253    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"]
Sep 11 14:19:38.327585 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:38.327563    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c6156e-96c8985d7-z7p2z"]
Sep 11 14:19:40.276689 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:40.276658    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cfca453-a275-465b-94fa-facb441835b1" path="/var/lib/kubelet/pods/3cfca453-a275-465b-94fa-facb441835b1/volumes"
Sep 11 14:19:43.303638 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:43.303594    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:19:48.300209 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:48.300180    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:19:52.964250 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:52.964164    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"]
Sep 11 14:19:52.964705 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:52.964543    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cfca453-a275-465b-94fa-facb441835b1" containerName="payload-processing"
Sep 11 14:19:52.964705 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:52.964555    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cfca453-a275-465b-94fa-facb441835b1" containerName="payload-processing"
Sep 11 14:19:52.964705 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:52.964641    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cfca453-a275-465b-94fa-facb441835b1" containerName="payload-processing"
Sep 11 14:19:52.969044 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:52.969024    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:52.972110 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:52.972090    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-1e2e7d-serving-cert\""
Sep 11 14:19:52.984558 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:52.984532    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"]
Sep 11 14:19:53.057677 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.057613    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.057677 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.057681    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.057915 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.057728    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.057915 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.057761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j7pb\" (UniqueName: \"kubernetes.io/projected/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-kube-api-access-5j7pb\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.158459 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.158423    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.158684 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.158471    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.158684 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.158530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.158684 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.158671    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j7pb\" (UniqueName: \"kubernetes.io/projected/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-kube-api-access-5j7pb\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.159213 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.159190    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.160966 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.160947    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.161054 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.161024    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.168699 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.168678    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j7pb\" (UniqueName: \"kubernetes.io/projected/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-kube-api-access-5j7pb\") pod \"maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.279694 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.279605    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:53.588411 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:53.588384    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"]
Sep 11 14:19:53.591096 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:19:53.591068    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef2b257_4e36_4d7d_a5a9_b6609f3a6957.slice/crio-65d3809aaee62909da3aadb0b1009ee47c8d5fd142dd883c47af6b7370dfdbd7 WatchSource:0}: Error finding container 65d3809aaee62909da3aadb0b1009ee47c8d5fd142dd883c47af6b7370dfdbd7: Status 404 returned error can't find the container with id 65d3809aaee62909da3aadb0b1009ee47c8d5fd142dd883c47af6b7370dfdbd7
Sep 11 14:19:54.377270 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:54.377224    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl" event={"ID":"fef2b257-4e36-4d7d-a5a9-b6609f3a6957","Type":"ContainerStarted","Data":"9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d"}
Sep 11 14:19:54.377772 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:54.377286    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl" event={"ID":"fef2b257-4e36-4d7d-a5a9-b6609f3a6957","Type":"ContainerStarted","Data":"65d3809aaee62909da3aadb0b1009ee47c8d5fd142dd883c47af6b7370dfdbd7"}
Sep 11 14:19:54.377772 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:54.377403    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:19:54.435694 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:19:54.435616    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl" podStartSLOduration=1.99167643 podStartE2EDuration="2.435588411s" totalImagesPullingTime="443.911981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:19:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:19:54.433690926 +0000 UTC m=+1344.771266488" watchObservedRunningTime="2026-09-11 14:19:54.435588411 +0000 UTC m=+1344.773163971"
Sep 11 14:20:00.375723 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.375694    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"]
Sep 11 14:20:00.379188 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.379169    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.388509 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.388488    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:20:00.391254 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.391237    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-335a68-serving-cert\""
Sep 11 14:20:00.420327 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.420301    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"]
Sep 11 14:20:00.525238 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.525200    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.525421 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.525247    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpkvk\" (UniqueName: \"kubernetes.io/projected/7e66a92b-b530-41fe-96ee-0350290a197a-kube-api-access-dpkvk\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.525421 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.525404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66a92b-b530-41fe-96ee-0350290a197a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.525551 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.525439    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.627042 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.626958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66a92b-b530-41fe-96ee-0350290a197a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.627042 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.626997    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.627042 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.627043    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.627283 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.627059    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpkvk\" (UniqueName: \"kubernetes.io/projected/7e66a92b-b530-41fe-96ee-0350290a197a-kube-api-access-dpkvk\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.627763 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.627737    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66a92b-b530-41fe-96ee-0350290a197a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.629591 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.629565    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.629704 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.629565    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.640068 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.640041    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpkvk\" (UniqueName: \"kubernetes.io/projected/7e66a92b-b530-41fe-96ee-0350290a197a-kube-api-access-dpkvk\") pod \"maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.691512 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.691472    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:00.971786 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.971707    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"]
Sep 11 14:20:00.974663 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:20:00.974609    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e66a92b_b530_41fe_96ee_0350290a197a.slice/crio-2ee01f3e1683719e737e525b53c0727ac0dfec5c5183459af63535984d6cdd3b WatchSource:0}: Error finding container 2ee01f3e1683719e737e525b53c0727ac0dfec5c5183459af63535984d6cdd3b: Status 404 returned error can't find the container with id 2ee01f3e1683719e737e525b53c0727ac0dfec5c5183459af63535984d6cdd3b
Sep 11 14:20:00.975894 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:00.975874    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:20:01.409982 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.409947    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn" event={"ID":"7e66a92b-b530-41fe-96ee-0350290a197a","Type":"ContainerStarted","Data":"2ee01f3e1683719e737e525b53c0727ac0dfec5c5183459af63535984d6cdd3b"}
Sep 11 14:20:01.567576 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.567543    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"]
Sep 11 14:20:01.570900 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.570885    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:01.574935 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.574919    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-335a68-dockercfg-lxvtm\""
Sep 11 14:20:01.575295 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.575276    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-335a68\""
Sep 11 14:20:01.597284 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.597258    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"]
Sep 11 14:20:01.737888 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.737815    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e206dd-7321-47a2-870e-f7090578f96a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx\" (UID: \"a7e206dd-7321-47a2-870e-f7090578f96a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:01.737888 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.737877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znb5t\" (UniqueName: \"kubernetes.io/projected/a7e206dd-7321-47a2-870e-f7090578f96a-kube-api-access-znb5t\") pod \"payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx\" (UID: \"a7e206dd-7321-47a2-870e-f7090578f96a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:01.839063 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.839030    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e206dd-7321-47a2-870e-f7090578f96a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx\" (UID: \"a7e206dd-7321-47a2-870e-f7090578f96a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:01.839235 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.839095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znb5t\" (UniqueName: \"kubernetes.io/projected/a7e206dd-7321-47a2-870e-f7090578f96a-kube-api-access-znb5t\") pod \"payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx\" (UID: \"a7e206dd-7321-47a2-870e-f7090578f96a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:01.839712 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.839693    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e206dd-7321-47a2-870e-f7090578f96a-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx\" (UID: \"a7e206dd-7321-47a2-870e-f7090578f96a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:01.865265 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.865236    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znb5t\" (UniqueName: \"kubernetes.io/projected/a7e206dd-7321-47a2-870e-f7090578f96a-kube-api-access-znb5t\") pod \"payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx\" (UID: \"a7e206dd-7321-47a2-870e-f7090578f96a\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:01.882218 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:01.882191    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:02.168588 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:02.168556    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"]
Sep 11 14:20:02.171465 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:20:02.171432    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e206dd_7321_47a2_870e_f7090578f96a.slice/crio-3f032feb44c0d5e309606150e93b228a734232417f593394206a3b8530d2ec63 WatchSource:0}: Error finding container 3f032feb44c0d5e309606150e93b228a734232417f593394206a3b8530d2ec63: Status 404 returned error can't find the container with id 3f032feb44c0d5e309606150e93b228a734232417f593394206a3b8530d2ec63
Sep 11 14:20:02.418088 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:02.418055    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx" event={"ID":"a7e206dd-7321-47a2-870e-f7090578f96a","Type":"ContainerStarted","Data":"b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19"}
Sep 11 14:20:02.418524 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:02.418115    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx" event={"ID":"a7e206dd-7321-47a2-870e-f7090578f96a","Type":"ContainerStarted","Data":"3f032feb44c0d5e309606150e93b228a734232417f593394206a3b8530d2ec63"}
Sep 11 14:20:02.418524 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:02.418232    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:02.419788 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:02.419713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn" event={"ID":"7e66a92b-b530-41fe-96ee-0350290a197a","Type":"ContainerStarted","Data":"03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300"}
Sep 11 14:20:02.419929 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:02.419841    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:02.455505 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:02.455458    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx" podStartSLOduration=1.455444425 podStartE2EDuration="1.455444425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:20:02.453191735 +0000 UTC m=+1352.790767294" watchObservedRunningTime="2026-09-11 14:20:02.455444425 +0000 UTC m=+1352.793020004"
Sep 11 14:20:02.489998 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:02.489956    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn" podStartSLOduration=1.886624062 podStartE2EDuration="2.489941086s" totalImagesPullingTime="603.317024ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:20:02.489741718 +0000 UTC m=+1352.827317291" watchObservedRunningTime="2026-09-11 14:20:02.489941086 +0000 UTC m=+1352.827516646"
Sep 11 14:20:08.429455 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:08.429426    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:20:13.426519 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:13.426489    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:20:15.442504 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.442463    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"]
Sep 11 14:20:15.447385 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.447355    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.450301 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.450278    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-031a56-serving-cert\""
Sep 11 14:20:15.461867 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.461843    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.461981 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.461871    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.461981 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.461898    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e8bf04-451a-40c0-a3e3-0827a1365196-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.461981 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.461928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9cv6\" (UniqueName: \"kubernetes.io/projected/81e8bf04-451a-40c0-a3e3-0827a1365196-kube-api-access-v9cv6\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.467105 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.467078    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"]
Sep 11 14:20:15.563035 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.562995    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9cv6\" (UniqueName: \"kubernetes.io/projected/81e8bf04-451a-40c0-a3e3-0827a1365196-kube-api-access-v9cv6\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.563217 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.563121    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.563217 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.563152    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.563217 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.563199    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e8bf04-451a-40c0-a3e3-0827a1365196-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.563385 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:20:15.563312    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-serving-cert: secret "maas-api-e2e-shared-b-w3-031a56-serving-cert" not found
Sep 11 14:20:15.563426 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:20:15.563397    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls podName:81e8bf04-451a-40c0-a3e3-0827a1365196 nodeName:}" failed. No retries permitted until 2026-09-11 14:20:16.063376198 +0000 UTC m=+1366.400951754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls") pod "maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2" (UID: "81e8bf04-451a-40c0-a3e3-0827a1365196") : secret "maas-api-e2e-shared-b-w3-031a56-serving-cert" not found
Sep 11 14:20:15.563978 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.563959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e8bf04-451a-40c0-a3e3-0827a1365196-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.565667 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.565644    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:15.579201 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:15.579172    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9cv6\" (UniqueName: \"kubernetes.io/projected/81e8bf04-451a-40c0-a3e3-0827a1365196-kube-api-access-v9cv6\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:16.066890 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.066845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:16.067052 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:20:16.066989    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-serving-cert: secret "maas-api-e2e-shared-b-w3-031a56-serving-cert" not found
Sep 11 14:20:16.067110 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:20:16.067060    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls podName:81e8bf04-451a-40c0-a3e3-0827a1365196 nodeName:}" failed. No retries permitted until 2026-09-11 14:20:17.067045304 +0000 UTC m=+1367.404620847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls") pod "maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2" (UID: "81e8bf04-451a-40c0-a3e3-0827a1365196") : secret "maas-api-e2e-shared-b-w3-031a56-serving-cert" not found
Sep 11 14:20:16.317608 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.317521    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"]
Sep 11 14:20:16.322206 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.322184    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:16.329929 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.329886    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-031a56-dockercfg-4rx5b\""
Sep 11 14:20:16.334200 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.334177    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-031a56\""
Sep 11 14:20:16.335036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.335016    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"]
Sep 11 14:20:16.368401 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.368374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgqhb\" (UniqueName: \"kubernetes.io/projected/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-kube-api-access-dgqhb\") pod \"payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9\" (UID: \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:16.368524 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.368407    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9\" (UID: \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:16.469689 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.469650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgqhb\" (UniqueName: \"kubernetes.io/projected/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-kube-api-access-dgqhb\") pod \"payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9\" (UID: \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:16.470152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.469709    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9\" (UID: \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:16.470589 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.470565    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9\" (UID: \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:16.479431 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.479407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgqhb\" (UniqueName: \"kubernetes.io/projected/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-kube-api-access-dgqhb\") pod \"payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9\" (UID: \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:16.634979 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.634901    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:16.909641 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:16.909597    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"]
Sep 11 14:20:16.911453 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:20:16.911422    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130aaa7d_5fc3_40db_bbea_21b0a4beda7c.slice/crio-19bf9d9fdb3032e560c2b580a1c20b9583bbba43ddf4b3f6054426d787bab30b WatchSource:0}: Error finding container 19bf9d9fdb3032e560c2b580a1c20b9583bbba43ddf4b3f6054426d787bab30b: Status 404 returned error can't find the container with id 19bf9d9fdb3032e560c2b580a1c20b9583bbba43ddf4b3f6054426d787bab30b
Sep 11 14:20:17.074864 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:17.074828    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:17.077289 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:17.077260    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:17.258163 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:17.258073    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:17.502222 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:17.502193    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9" event={"ID":"130aaa7d-5fc3-40db-bbea-21b0a4beda7c","Type":"ContainerStarted","Data":"a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8"}
Sep 11 14:20:17.502592 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:17.502242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9" event={"ID":"130aaa7d-5fc3-40db-bbea-21b0a4beda7c","Type":"ContainerStarted","Data":"19bf9d9fdb3032e560c2b580a1c20b9583bbba43ddf4b3f6054426d787bab30b"}
Sep 11 14:20:17.502592 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:17.502288    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:17.543133 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:17.543045    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9" podStartSLOduration=1.543029549 podStartE2EDuration="1.543029549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:20:17.542686056 +0000 UTC m=+1367.880261616" watchObservedRunningTime="2026-09-11 14:20:17.543029549 +0000 UTC m=+1367.880605155"
Sep 11 14:20:17.554701 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:17.554471    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"]
Sep 11 14:20:17.558090 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:20:17.558062    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e8bf04_451a_40c0_a3e3_0827a1365196.slice/crio-ffd29274114c8965cd16405a01fa269a21239dd3ef4823577fe8132c9beda168 WatchSource:0}: Error finding container ffd29274114c8965cd16405a01fa269a21239dd3ef4823577fe8132c9beda168: Status 404 returned error can't find the container with id ffd29274114c8965cd16405a01fa269a21239dd3ef4823577fe8132c9beda168
Sep 11 14:20:18.508461 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:18.508374    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2" event={"ID":"81e8bf04-451a-40c0-a3e3-0827a1365196","Type":"ContainerStarted","Data":"5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1"}
Sep 11 14:20:18.508461 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:18.508427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2" event={"ID":"81e8bf04-451a-40c0-a3e3-0827a1365196","Type":"ContainerStarted","Data":"ffd29274114c8965cd16405a01fa269a21239dd3ef4823577fe8132c9beda168"}
Sep 11 14:20:18.508461 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:18.508460    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:18.538359 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:18.538311    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2" podStartSLOduration=3.046131026 podStartE2EDuration="3.53829538s" totalImagesPullingTime="492.164354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:20:18.537295221 +0000 UTC m=+1368.874870780" watchObservedRunningTime="2026-09-11 14:20:18.53829538 +0000 UTC m=+1368.875870939"
Sep 11 14:20:24.518850 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:24.518821    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:20:28.510581 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:28.510556    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:20:33.004098 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.004060    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"]
Sep 11 14:20:33.008370 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.008351    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:20:33.012853 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.012833    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-557930d3-dockercfg-l288l\""
Sep 11 14:20:33.014237 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.014218    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-557930d3\""
Sep 11 14:20:33.086795 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.086770    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"]
Sep 11 14:20:33.117501 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.117474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15693360-d55f-4a6b-8afb-5711829324df-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz\" (UID: \"15693360-d55f-4a6b-8afb-5711829324df\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:20:33.117619 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.117524    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ss66\" (UniqueName: \"kubernetes.io/projected/15693360-d55f-4a6b-8afb-5711829324df-kube-api-access-2ss66\") pod \"payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz\" (UID: \"15693360-d55f-4a6b-8afb-5711829324df\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:20:33.218676 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.218644    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ss66\" (UniqueName: \"kubernetes.io/projected/15693360-d55f-4a6b-8afb-5711829324df-kube-api-access-2ss66\") pod \"payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz\" (UID: \"15693360-d55f-4a6b-8afb-5711829324df\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:20:33.218831 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.218743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15693360-d55f-4a6b-8afb-5711829324df-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz\" (UID: \"15693360-d55f-4a6b-8afb-5711829324df\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:20:33.219377 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.219359    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15693360-d55f-4a6b-8afb-5711829324df-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz\" (UID: \"15693360-d55f-4a6b-8afb-5711829324df\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:20:33.236863 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.236840    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ss66\" (UniqueName: \"kubernetes.io/projected/15693360-d55f-4a6b-8afb-5711829324df-kube-api-access-2ss66\") pod \"payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz\" (UID: \"15693360-d55f-4a6b-8afb-5711829324df\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:20:33.317983 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.317953    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:20:33.599986 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:20:33.599943    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15693360_d55f_4a6b_8afb_5711829324df.slice/crio-8a1bb0c9f2e923b7d3676b8d70ec61490c665d3e81c0853cac66f64f904ae4bc WatchSource:0}: Error finding container 8a1bb0c9f2e923b7d3676b8d70ec61490c665d3e81c0853cac66f64f904ae4bc: Status 404 returned error can't find the container with id 8a1bb0c9f2e923b7d3676b8d70ec61490c665d3e81c0853cac66f64f904ae4bc
Sep 11 14:20:33.600089 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:33.600000    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"]
Sep 11 14:20:34.592252 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:34.592216    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz" event={"ID":"15693360-d55f-4a6b-8afb-5711829324df","Type":"ContainerStarted","Data":"1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505"}
Sep 11 14:20:34.592680 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:34.592286    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:20:34.592680 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:34.592309    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz" event={"ID":"15693360-d55f-4a6b-8afb-5711829324df","Type":"ContainerStarted","Data":"8a1bb0c9f2e923b7d3676b8d70ec61490c665d3e81c0853cac66f64f904ae4bc"}
Sep 11 14:20:34.613527 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:34.613481    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz" podStartSLOduration=2.613465884 podStartE2EDuration="2.613465884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:20:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:20:34.611499902 +0000 UTC m=+1384.949075461" watchObservedRunningTime="2026-09-11 14:20:34.613465884 +0000 UTC m=+1384.951041477"
Sep 11 14:20:45.599028 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:20:45.598998    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:21:30.308399 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:21:30.308333    2564 scope.go:117] "RemoveContainer" containerID="1fabb0209744097e83498f5bcb76e5860cfa97f2d9c48d7bde7178d5c51870d1"
Sep 11 14:22:14.700294 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:14.700256    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"]
Sep 11 14:22:14.700794 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:14.700517    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz" podUID="15693360-d55f-4a6b-8afb-5711829324df" containerName="payload-pre-processing" containerID="cri-o://1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505" gracePeriod=30
Sep 11 14:22:14.951485 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:14.951424    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:22:15.061674 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.061616    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ss66\" (UniqueName: \"kubernetes.io/projected/15693360-d55f-4a6b-8afb-5711829324df-kube-api-access-2ss66\") pod \"15693360-d55f-4a6b-8afb-5711829324df\" (UID: \"15693360-d55f-4a6b-8afb-5711829324df\") "
Sep 11 14:22:15.061841 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.061710    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15693360-d55f-4a6b-8afb-5711829324df-plugins-config-volume\") pod \"15693360-d55f-4a6b-8afb-5711829324df\" (UID: \"15693360-d55f-4a6b-8afb-5711829324df\") "
Sep 11 14:22:15.062085 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.062061    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15693360-d55f-4a6b-8afb-5711829324df-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "15693360-d55f-4a6b-8afb-5711829324df" (UID: "15693360-d55f-4a6b-8afb-5711829324df"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:22:15.063821 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.063790    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15693360-d55f-4a6b-8afb-5711829324df-kube-api-access-2ss66" (OuterVolumeSpecName: "kube-api-access-2ss66") pod "15693360-d55f-4a6b-8afb-5711829324df" (UID: "15693360-d55f-4a6b-8afb-5711829324df"). InnerVolumeSpecName "kube-api-access-2ss66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:22:15.107978 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.107945    2564 generic.go:356] "Generic (PLEG): container finished" podID="15693360-d55f-4a6b-8afb-5711829324df" containerID="1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505" exitCode=0
Sep 11 14:22:15.108122 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.107989    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"
Sep 11 14:22:15.108122 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.108032    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz" event={"ID":"15693360-d55f-4a6b-8afb-5711829324df","Type":"ContainerDied","Data":"1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505"}
Sep 11 14:22:15.108122 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.108081    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz" event={"ID":"15693360-d55f-4a6b-8afb-5711829324df","Type":"ContainerDied","Data":"8a1bb0c9f2e923b7d3676b8d70ec61490c665d3e81c0853cac66f64f904ae4bc"}
Sep 11 14:22:15.108122 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.108104    2564 scope.go:117] "RemoveContainer" containerID="1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505"
Sep 11 14:22:15.117044 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.117027    2564 scope.go:117] "RemoveContainer" containerID="1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505"
Sep 11 14:22:15.117300 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:22:15.117277    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505\": container with ID starting with 1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505 not found: ID does not exist" containerID="1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505"
Sep 11 14:22:15.117346 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.117312    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505"} err="failed to get container status \"1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505\": rpc error: code = NotFound desc = could not find container \"1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505\": container with ID starting with 1ab931f54ceed1b8a45848efcada0e4f6a12f211481d9b95f5001462654cd505 not found: ID does not exist"
Sep 11 14:22:15.161356 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.161328    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"]
Sep 11 14:22:15.163429 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.163411    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ss66\" (UniqueName: \"kubernetes.io/projected/15693360-d55f-4a6b-8afb-5711829324df-kube-api-access-2ss66\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:22:15.163485 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.163433    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/15693360-d55f-4a6b-8afb-5711829324df-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:22:15.174219 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:15.174197    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-557930d3-56f9db478b-zmphz"]
Sep 11 14:22:16.275819 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:22:16.275784    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15693360-d55f-4a6b-8afb-5711829324df" path="/var/lib/kubelet/pods/15693360-d55f-4a6b-8afb-5711829324df/volumes"
Sep 11 14:24:14.468780 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.468747    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"]
Sep 11 14:24:14.469192 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.469179    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15693360-d55f-4a6b-8afb-5711829324df" containerName="payload-pre-processing"
Sep 11 14:24:14.469236 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.469194    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="15693360-d55f-4a6b-8afb-5711829324df" containerName="payload-pre-processing"
Sep 11 14:24:14.469284 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.469275    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="15693360-d55f-4a6b-8afb-5711829324df" containerName="payload-pre-processing"
Sep 11 14:24:14.472471 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.472447    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.477092 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.477067    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d52edb-serving-cert\""
Sep 11 14:24:14.488834 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.488809    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"]
Sep 11 14:24:14.543900 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.543864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-metrics-certs\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.544104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.543925    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-maas-api-tls\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.544104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.544043    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4257913-14d8-4f52-b7ef-6b5384331c2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.544104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.544095    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz2zs\" (UniqueName: \"kubernetes.io/projected/b4257913-14d8-4f52-b7ef-6b5384331c2c-kube-api-access-jz2zs\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.645050 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.645011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4257913-14d8-4f52-b7ef-6b5384331c2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.645050 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.645057    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz2zs\" (UniqueName: \"kubernetes.io/projected/b4257913-14d8-4f52-b7ef-6b5384331c2c-kube-api-access-jz2zs\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.645319 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.645106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-metrics-certs\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.645319 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.645165    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-maas-api-tls\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.645319 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:24:14.645276    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-serving-cert: secret "maas-api-e2e-api-a-d52edb-serving-cert" not found
Sep 11 14:24:14.645489 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:24:14.645337    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-maas-api-tls podName:b4257913-14d8-4f52-b7ef-6b5384331c2c nodeName:}" failed. No retries permitted until 2026-09-11 14:24:15.145318382 +0000 UTC m=+1605.482893937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-maas-api-tls") pod "maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z" (UID: "b4257913-14d8-4f52-b7ef-6b5384331c2c") : secret "maas-api-e2e-api-a-d52edb-serving-cert" not found
Sep 11 14:24:14.645814 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.645789    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4257913-14d8-4f52-b7ef-6b5384331c2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.647537 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.647518    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-metrics-certs\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:14.657351 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:14.657324    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz2zs\" (UniqueName: \"kubernetes.io/projected/b4257913-14d8-4f52-b7ef-6b5384331c2c-kube-api-access-jz2zs\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:15.150255 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.150214    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-maas-api-tls\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:15.152757 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.152734    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-maas-api-tls\") pod \"maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:15.384408 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.384370    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:15.425917 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.425886    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"]
Sep 11 14:24:15.429753 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.429728    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:24:15.432885 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.432862    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-d52edb\""
Sep 11 14:24:15.433019 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.432916    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-d52edb-dockercfg-cghzm\""
Sep 11 14:24:15.446326 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.446302    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"]
Sep 11 14:24:15.555679 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.555620    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9966e7e9-6310-49de-9dbc-c25df5cdafe2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt\" (UID: \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:24:15.556031 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.555778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xh75\" (UniqueName: \"kubernetes.io/projected/9966e7e9-6310-49de-9dbc-c25df5cdafe2-kube-api-access-9xh75\") pod \"payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt\" (UID: \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:24:15.657157 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.657068    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9966e7e9-6310-49de-9dbc-c25df5cdafe2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt\" (UID: \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:24:15.657311 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.657166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xh75\" (UniqueName: \"kubernetes.io/projected/9966e7e9-6310-49de-9dbc-c25df5cdafe2-kube-api-access-9xh75\") pod \"payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt\" (UID: \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:24:15.657909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.657888    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9966e7e9-6310-49de-9dbc-c25df5cdafe2-plugins-config-volume\") pod \"payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt\" (UID: \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:24:15.666992 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.666969    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xh75\" (UniqueName: \"kubernetes.io/projected/9966e7e9-6310-49de-9dbc-c25df5cdafe2-kube-api-access-9xh75\") pod \"payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt\" (UID: \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\") " pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:24:15.681101 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.681082    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"]
Sep 11 14:24:15.683023 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:24:15.682995    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4257913_14d8_4f52_b7ef_6b5384331c2c.slice/crio-4e50dbee562c168ada0cd7c4a2ba85124fdd0ef8865c17a2fc2879befab19d57 WatchSource:0}: Error finding container 4e50dbee562c168ada0cd7c4a2ba85124fdd0ef8865c17a2fc2879befab19d57: Status 404 returned error can't find the container with id 4e50dbee562c168ada0cd7c4a2ba85124fdd0ef8865c17a2fc2879befab19d57
Sep 11 14:24:15.711674 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.711644    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z" event={"ID":"b4257913-14d8-4f52-b7ef-6b5384331c2c","Type":"ContainerStarted","Data":"4e50dbee562c168ada0cd7c4a2ba85124fdd0ef8865c17a2fc2879befab19d57"}
Sep 11 14:24:15.746507 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:15.746484    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:24:16.026105 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:16.026034    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"]
Sep 11 14:24:16.029611 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:24:16.029587    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9966e7e9_6310_49de_9dbc_c25df5cdafe2.slice/crio-53b9d1cef36268b0b9605b4be4d839acfe8ef511c7c6111e6ded04763db16440 WatchSource:0}: Error finding container 53b9d1cef36268b0b9605b4be4d839acfe8ef511c7c6111e6ded04763db16440: Status 404 returned error can't find the container with id 53b9d1cef36268b0b9605b4be4d839acfe8ef511c7c6111e6ded04763db16440
Sep 11 14:24:16.719843 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:16.719796    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z" event={"ID":"b4257913-14d8-4f52-b7ef-6b5384331c2c","Type":"ContainerStarted","Data":"b1f3d73a13ce6c87999bb9b38d4d41bc289cae2d8b7c270f246476cf797954bb"}
Sep 11 14:24:16.720359 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:16.720329    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:16.721725 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:16.721698    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt" event={"ID":"9966e7e9-6310-49de-9dbc-c25df5cdafe2","Type":"ContainerStarted","Data":"988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7"}
Sep 11 14:24:16.721854 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:16.721745    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt" event={"ID":"9966e7e9-6310-49de-9dbc-c25df5cdafe2","Type":"ContainerStarted","Data":"53b9d1cef36268b0b9605b4be4d839acfe8ef511c7c6111e6ded04763db16440"}
Sep 11 14:24:16.721854 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:16.721811    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:24:16.739065 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:16.739022    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z" podStartSLOduration=2.135652279 podStartE2EDuration="2.739008972s" totalImagesPullingTime="603.356693ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:24:16.73775973 +0000 UTC m=+1607.075335304" watchObservedRunningTime="2026-09-11 14:24:16.739008972 +0000 UTC m=+1607.076584531"
Sep 11 14:24:16.756547 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:16.756501    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt" podStartSLOduration=1.756486837 podStartE2EDuration="1.756486837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:24:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:24:16.755312734 +0000 UTC m=+1607.092888295" watchObservedRunningTime="2026-09-11 14:24:16.756486837 +0000 UTC m=+1607.094062457"
Sep 11 14:24:22.732398 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:22.732371    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:24:27.729484 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:24:27.729448    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:25:38.852275 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:38.852236    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"]
Sep 11 14:25:38.852801 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:38.852459    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z" podUID="b4257913-14d8-4f52-b7ef-6b5384331c2c" containerName="maas-api" containerID="cri-o://b1f3d73a13ce6c87999bb9b38d4d41bc289cae2d8b7c270f246476cf797954bb" gracePeriod=30
Sep 11 14:25:39.033740 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.033701    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"]
Sep 11 14:25:39.034003 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.033943    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt" podUID="9966e7e9-6310-49de-9dbc-c25df5cdafe2" containerName="payload-processing" containerID="cri-o://988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7" gracePeriod=30
Sep 11 14:25:39.147089 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.147017    2564 generic.go:356] "Generic (PLEG): container finished" podID="b4257913-14d8-4f52-b7ef-6b5384331c2c" containerID="b1f3d73a13ce6c87999bb9b38d4d41bc289cae2d8b7c270f246476cf797954bb" exitCode=0
Sep 11 14:25:39.147145 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.147088    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z" event={"ID":"b4257913-14d8-4f52-b7ef-6b5384331c2c","Type":"ContainerDied","Data":"b1f3d73a13ce6c87999bb9b38d4d41bc289cae2d8b7c270f246476cf797954bb"}
Sep 11 14:25:39.147145 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.147141    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z" event={"ID":"b4257913-14d8-4f52-b7ef-6b5384331c2c","Type":"ContainerDied","Data":"4e50dbee562c168ada0cd7c4a2ba85124fdd0ef8865c17a2fc2879befab19d57"}
Sep 11 14:25:39.147252 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.147159    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e50dbee562c168ada0cd7c4a2ba85124fdd0ef8865c17a2fc2879befab19d57"
Sep 11 14:25:39.161820 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.161797    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:25:39.224861 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.224834    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-maas-api-tls\") pod \"b4257913-14d8-4f52-b7ef-6b5384331c2c\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") "
Sep 11 14:25:39.225016 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.224958    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-metrics-certs\") pod \"b4257913-14d8-4f52-b7ef-6b5384331c2c\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") "
Sep 11 14:25:39.225118 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.225018    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4257913-14d8-4f52-b7ef-6b5384331c2c-openshift-service-ca\") pod \"b4257913-14d8-4f52-b7ef-6b5384331c2c\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") "
Sep 11 14:25:39.225118 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.225047    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz2zs\" (UniqueName: \"kubernetes.io/projected/b4257913-14d8-4f52-b7ef-6b5384331c2c-kube-api-access-jz2zs\") pod \"b4257913-14d8-4f52-b7ef-6b5384331c2c\" (UID: \"b4257913-14d8-4f52-b7ef-6b5384331c2c\") "
Sep 11 14:25:39.226095 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.225972    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4257913-14d8-4f52-b7ef-6b5384331c2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4257913-14d8-4f52-b7ef-6b5384331c2c" (UID: "b4257913-14d8-4f52-b7ef-6b5384331c2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:25:39.227618 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.227579    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4257913-14d8-4f52-b7ef-6b5384331c2c" (UID: "b4257913-14d8-4f52-b7ef-6b5384331c2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:25:39.231682 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.230738    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4257913-14d8-4f52-b7ef-6b5384331c2c-kube-api-access-jz2zs" (OuterVolumeSpecName: "kube-api-access-jz2zs") pod "b4257913-14d8-4f52-b7ef-6b5384331c2c" (UID: "b4257913-14d8-4f52-b7ef-6b5384331c2c"). InnerVolumeSpecName "kube-api-access-jz2zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:25:39.249993 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.248449    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b4257913-14d8-4f52-b7ef-6b5384331c2c" (UID: "b4257913-14d8-4f52-b7ef-6b5384331c2c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:25:39.289112 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.289087    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:25:39.326689 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.326664    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9966e7e9-6310-49de-9dbc-c25df5cdafe2-plugins-config-volume\") pod \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\" (UID: \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\") "
Sep 11 14:25:39.326826 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.326716    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xh75\" (UniqueName: \"kubernetes.io/projected/9966e7e9-6310-49de-9dbc-c25df5cdafe2-kube-api-access-9xh75\") pod \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\" (UID: \"9966e7e9-6310-49de-9dbc-c25df5cdafe2\") "
Sep 11 14:25:39.326968 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.326952    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:25:39.327007 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.326974    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4257913-14d8-4f52-b7ef-6b5384331c2c-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:25:39.327039 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.327012    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4257913-14d8-4f52-b7ef-6b5384331c2c-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:25:39.327039 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.327027    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz2zs\" (UniqueName: \"kubernetes.io/projected/b4257913-14d8-4f52-b7ef-6b5384331c2c-kube-api-access-jz2zs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:25:39.327103 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.327022    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9966e7e9-6310-49de-9dbc-c25df5cdafe2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9966e7e9-6310-49de-9dbc-c25df5cdafe2" (UID: "9966e7e9-6310-49de-9dbc-c25df5cdafe2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:25:39.328858 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.328838    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9966e7e9-6310-49de-9dbc-c25df5cdafe2-kube-api-access-9xh75" (OuterVolumeSpecName: "kube-api-access-9xh75") pod "9966e7e9-6310-49de-9dbc-c25df5cdafe2" (UID: "9966e7e9-6310-49de-9dbc-c25df5cdafe2"). InnerVolumeSpecName "kube-api-access-9xh75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:25:39.428096 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.428018    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9966e7e9-6310-49de-9dbc-c25df5cdafe2-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:25:39.428096 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:39.428050    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xh75\" (UniqueName: \"kubernetes.io/projected/9966e7e9-6310-49de-9dbc-c25df5cdafe2-kube-api-access-9xh75\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:25:40.153442 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.153355    2564 generic.go:356] "Generic (PLEG): container finished" podID="9966e7e9-6310-49de-9dbc-c25df5cdafe2" containerID="988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7" exitCode=0
Sep 11 14:25:40.153442 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.153402    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"
Sep 11 14:25:40.153906 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.153443    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt" event={"ID":"9966e7e9-6310-49de-9dbc-c25df5cdafe2","Type":"ContainerDied","Data":"988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7"}
Sep 11 14:25:40.153906 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.153496    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt" event={"ID":"9966e7e9-6310-49de-9dbc-c25df5cdafe2","Type":"ContainerDied","Data":"53b9d1cef36268b0b9605b4be4d839acfe8ef511c7c6111e6ded04763db16440"}
Sep 11 14:25:40.153906 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.153518    2564 scope.go:117] "RemoveContainer" containerID="988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7"
Sep 11 14:25:40.153906 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.153652    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"
Sep 11 14:25:40.163573 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.163547    2564 scope.go:117] "RemoveContainer" containerID="988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7"
Sep 11 14:25:40.163953 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:25:40.163917    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7\": container with ID starting with 988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7 not found: ID does not exist" containerID="988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7"
Sep 11 14:25:40.164046 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.163954    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7"} err="failed to get container status \"988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7\": rpc error: code = NotFound desc = could not find container \"988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7\": container with ID starting with 988c77e29f8dbaf5f83e502798bbd0dc1d742675997cba42f0edb95a853014d7 not found: ID does not exist"
Sep 11 14:25:40.182399 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.182373    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"]
Sep 11 14:25:40.189012 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.188990    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d52edb-84859dfbcd-k9m9z"]
Sep 11 14:25:40.218343 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.218318    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"]
Sep 11 14:25:40.224332 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.224311    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-d52edb-58ff7fb7f9-b64vt"]
Sep 11 14:25:40.275969 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.275933    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9966e7e9-6310-49de-9dbc-c25df5cdafe2" path="/var/lib/kubelet/pods/9966e7e9-6310-49de-9dbc-c25df5cdafe2/volumes"
Sep 11 14:25:40.276405 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:25:40.276386    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4257913-14d8-4f52-b7ef-6b5384331c2c" path="/var/lib/kubelet/pods/b4257913-14d8-4f52-b7ef-6b5384331c2c/volumes"
Sep 11 14:26:22.328405 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.328370    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"]
Sep 11 14:26:22.328896 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.328565    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh" podUID="b4e68cc5-2e69-435c-abd6-4f36652d9b19" containerName="maas-api" containerID="cri-o://c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9" gracePeriod=30
Sep 11 14:26:22.417437 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.417404    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"]
Sep 11 14:26:22.418721 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.417807    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g" podUID="8734bbaa-bc41-4721-8e5b-ee73aad405f0" containerName="payload-pre-processing" containerID="cri-o://b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa" gracePeriod=30
Sep 11 14:26:22.669043 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.669019    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:26:22.688191 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.688167    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:26:22.732287 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.732249    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdx8k\" (UniqueName: \"kubernetes.io/projected/b4e68cc5-2e69-435c-abd6-4f36652d9b19-kube-api-access-fdx8k\") pod \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") "
Sep 11 14:26:22.732287 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.732291    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e68cc5-2e69-435c-abd6-4f36652d9b19-openshift-service-ca\") pod \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") "
Sep 11 14:26:22.732521 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.732327    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-maas-api-tls\") pod \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") "
Sep 11 14:26:22.732521 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.732359    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwn2v\" (UniqueName: \"kubernetes.io/projected/8734bbaa-bc41-4721-8e5b-ee73aad405f0-kube-api-access-mwn2v\") pod \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\" (UID: \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\") "
Sep 11 14:26:22.732521 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.732430    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-metrics-certs\") pod \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\" (UID: \"b4e68cc5-2e69-435c-abd6-4f36652d9b19\") "
Sep 11 14:26:22.732521 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.732455    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8734bbaa-bc41-4721-8e5b-ee73aad405f0-plugins-config-volume\") pod \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\" (UID: \"8734bbaa-bc41-4721-8e5b-ee73aad405f0\") "
Sep 11 14:26:22.732774 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.732738    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4e68cc5-2e69-435c-abd6-4f36652d9b19-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4e68cc5-2e69-435c-abd6-4f36652d9b19" (UID: "b4e68cc5-2e69-435c-abd6-4f36652d9b19"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:26:22.733102 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.733077    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8734bbaa-bc41-4721-8e5b-ee73aad405f0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8734bbaa-bc41-4721-8e5b-ee73aad405f0" (UID: "8734bbaa-bc41-4721-8e5b-ee73aad405f0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:26:22.735146 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.735114    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4e68cc5-2e69-435c-abd6-4f36652d9b19" (UID: "b4e68cc5-2e69-435c-abd6-4f36652d9b19"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:26:22.735286 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.735267    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b4e68cc5-2e69-435c-abd6-4f36652d9b19" (UID: "b4e68cc5-2e69-435c-abd6-4f36652d9b19"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:26:22.735370 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.735315    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e68cc5-2e69-435c-abd6-4f36652d9b19-kube-api-access-fdx8k" (OuterVolumeSpecName: "kube-api-access-fdx8k") pod "b4e68cc5-2e69-435c-abd6-4f36652d9b19" (UID: "b4e68cc5-2e69-435c-abd6-4f36652d9b19"). InnerVolumeSpecName "kube-api-access-fdx8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:26:22.735370 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.735341    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8734bbaa-bc41-4721-8e5b-ee73aad405f0-kube-api-access-mwn2v" (OuterVolumeSpecName: "kube-api-access-mwn2v") pod "8734bbaa-bc41-4721-8e5b-ee73aad405f0" (UID: "8734bbaa-bc41-4721-8e5b-ee73aad405f0"). InnerVolumeSpecName "kube-api-access-mwn2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:26:22.834155 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.834121    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:22.834155 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.834155    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwn2v\" (UniqueName: \"kubernetes.io/projected/8734bbaa-bc41-4721-8e5b-ee73aad405f0-kube-api-access-mwn2v\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:22.834344 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.834175    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4e68cc5-2e69-435c-abd6-4f36652d9b19-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:22.834344 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.834184    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8734bbaa-bc41-4721-8e5b-ee73aad405f0-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:22.834344 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.834193    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdx8k\" (UniqueName: \"kubernetes.io/projected/b4e68cc5-2e69-435c-abd6-4f36652d9b19-kube-api-access-fdx8k\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:22.834344 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:22.834202    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4e68cc5-2e69-435c-abd6-4f36652d9b19-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:23.372072 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.372039    2564 generic.go:356] "Generic (PLEG): container finished" podID="b4e68cc5-2e69-435c-abd6-4f36652d9b19" containerID="c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9" exitCode=0
Sep 11 14:26:23.372461 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.372084    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"
Sep 11 14:26:23.372461 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.372120    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh" event={"ID":"b4e68cc5-2e69-435c-abd6-4f36652d9b19","Type":"ContainerDied","Data":"c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9"}
Sep 11 14:26:23.372461 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.372189    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh" event={"ID":"b4e68cc5-2e69-435c-abd6-4f36652d9b19","Type":"ContainerDied","Data":"8a3d7f3e75003aa07b947b3772b7c1c133db244a1ef339aec3bda908fe329a6d"}
Sep 11 14:26:23.372461 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.372222    2564 scope.go:117] "RemoveContainer" containerID="c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9"
Sep 11 14:26:23.373673 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.373553    2564 generic.go:356] "Generic (PLEG): container finished" podID="8734bbaa-bc41-4721-8e5b-ee73aad405f0" containerID="b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa" exitCode=0
Sep 11 14:26:23.373673 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.373582    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g" event={"ID":"8734bbaa-bc41-4721-8e5b-ee73aad405f0","Type":"ContainerDied","Data":"b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa"}
Sep 11 14:26:23.373673 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.373599    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"
Sep 11 14:26:23.373673 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.373610    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g" event={"ID":"8734bbaa-bc41-4721-8e5b-ee73aad405f0","Type":"ContainerDied","Data":"7b434e2dbf61a9938f470e27eb08e8695a0cf663b55884b5946c815cd863d32d"}
Sep 11 14:26:23.382475 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.382459    2564 scope.go:117] "RemoveContainer" containerID="c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9"
Sep 11 14:26:23.382788 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:26:23.382764    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9\": container with ID starting with c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9 not found: ID does not exist" containerID="c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9"
Sep 11 14:26:23.382881 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.382800    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9"} err="failed to get container status \"c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9\": rpc error: code = NotFound desc = could not find container \"c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9\": container with ID starting with c9370779b6cb41aa93d1b0d3e0187fba6c48a49685733ca7ac5ed8d77931e1b9 not found: ID does not exist"
Sep 11 14:26:23.382881 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.382821    2564 scope.go:117] "RemoveContainer" containerID="b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa"
Sep 11 14:26:23.390854 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.390836    2564 scope.go:117] "RemoveContainer" containerID="b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa"
Sep 11 14:26:23.391103 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:26:23.391087    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa\": container with ID starting with b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa not found: ID does not exist" containerID="b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa"
Sep 11 14:26:23.391144 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.391111    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa"} err="failed to get container status \"b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa\": rpc error: code = NotFound desc = could not find container \"b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa\": container with ID starting with b780786ecb022039056b0f606973a5f53045608853657ef099bbf36521b722aa not found: ID does not exist"
Sep 11 14:26:23.399540 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.399520    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"]
Sep 11 14:26:23.405590 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.405566    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fb993c-d98bc5f74-hml4g"]
Sep 11 14:26:23.421289 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.421259    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"]
Sep 11 14:26:23.425958 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:23.425939    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-fb993c-5578d8544c-hznrh"]
Sep 11 14:26:24.280844 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:24.277005    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8734bbaa-bc41-4721-8e5b-ee73aad405f0" path="/var/lib/kubelet/pods/8734bbaa-bc41-4721-8e5b-ee73aad405f0/volumes"
Sep 11 14:26:24.280844 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:24.277645    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4e68cc5-2e69-435c-abd6-4f36652d9b19" path="/var/lib/kubelet/pods/b4e68cc5-2e69-435c-abd6-4f36652d9b19/volumes"
Sep 11 14:26:44.593418 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:44.593386    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"]
Sep 11 14:26:44.593853 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:44.593588    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn" podUID="7e66a92b-b530-41fe-96ee-0350290a197a" containerName="maas-api" containerID="cri-o://03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300" gracePeriod=30
Sep 11 14:26:44.677143 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:44.677106    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"]
Sep 11 14:26:44.677415 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:44.677360    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx" podUID="a7e206dd-7321-47a2-870e-f7090578f96a" containerName="payload-processing" containerID="cri-o://b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19" gracePeriod=30
Sep 11 14:26:44.927360 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:44.927333    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:26:44.951745 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:44.951695    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:26:45.027389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.027357    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znb5t\" (UniqueName: \"kubernetes.io/projected/a7e206dd-7321-47a2-870e-f7090578f96a-kube-api-access-znb5t\") pod \"a7e206dd-7321-47a2-870e-f7090578f96a\" (UID: \"a7e206dd-7321-47a2-870e-f7090578f96a\") "
Sep 11 14:26:45.027541 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.027425    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-maas-api-tls\") pod \"7e66a92b-b530-41fe-96ee-0350290a197a\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") "
Sep 11 14:26:45.027541 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.027454    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66a92b-b530-41fe-96ee-0350290a197a-openshift-service-ca\") pod \"7e66a92b-b530-41fe-96ee-0350290a197a\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") "
Sep 11 14:26:45.027541 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.027474    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-metrics-certs\") pod \"7e66a92b-b530-41fe-96ee-0350290a197a\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") "
Sep 11 14:26:45.027541 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.027513    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpkvk\" (UniqueName: \"kubernetes.io/projected/7e66a92b-b530-41fe-96ee-0350290a197a-kube-api-access-dpkvk\") pod \"7e66a92b-b530-41fe-96ee-0350290a197a\" (UID: \"7e66a92b-b530-41fe-96ee-0350290a197a\") "
Sep 11 14:26:45.027820 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.027547    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e206dd-7321-47a2-870e-f7090578f96a-plugins-config-volume\") pod \"a7e206dd-7321-47a2-870e-f7090578f96a\" (UID: \"a7e206dd-7321-47a2-870e-f7090578f96a\") "
Sep 11 14:26:45.027893 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.027859    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e66a92b-b530-41fe-96ee-0350290a197a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e66a92b-b530-41fe-96ee-0350290a197a" (UID: "7e66a92b-b530-41fe-96ee-0350290a197a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:26:45.028014 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.027980    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7e206dd-7321-47a2-870e-f7090578f96a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7e206dd-7321-47a2-870e-f7090578f96a" (UID: "a7e206dd-7321-47a2-870e-f7090578f96a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:26:45.029748 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.029716    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e206dd-7321-47a2-870e-f7090578f96a-kube-api-access-znb5t" (OuterVolumeSpecName: "kube-api-access-znb5t") pod "a7e206dd-7321-47a2-870e-f7090578f96a" (UID: "a7e206dd-7321-47a2-870e-f7090578f96a"). InnerVolumeSpecName "kube-api-access-znb5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:26:45.029862 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.029764    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e66a92b-b530-41fe-96ee-0350290a197a-kube-api-access-dpkvk" (OuterVolumeSpecName: "kube-api-access-dpkvk") pod "7e66a92b-b530-41fe-96ee-0350290a197a" (UID: "7e66a92b-b530-41fe-96ee-0350290a197a"). InnerVolumeSpecName "kube-api-access-dpkvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:26:45.029862 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.029821    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7e66a92b-b530-41fe-96ee-0350290a197a" (UID: "7e66a92b-b530-41fe-96ee-0350290a197a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:26:45.029862 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.029835    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e66a92b-b530-41fe-96ee-0350290a197a" (UID: "7e66a92b-b530-41fe-96ee-0350290a197a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:26:45.128740 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.128702    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpkvk\" (UniqueName: \"kubernetes.io/projected/7e66a92b-b530-41fe-96ee-0350290a197a-kube-api-access-dpkvk\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:45.128740 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.128737    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e206dd-7321-47a2-870e-f7090578f96a-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:45.128966 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.128752    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znb5t\" (UniqueName: \"kubernetes.io/projected/a7e206dd-7321-47a2-870e-f7090578f96a-kube-api-access-znb5t\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:45.128966 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.128766    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:45.128966 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.128779    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e66a92b-b530-41fe-96ee-0350290a197a-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:45.128966 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.128792    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e66a92b-b530-41fe-96ee-0350290a197a-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:26:45.486828 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.486788    2564 generic.go:356] "Generic (PLEG): container finished" podID="7e66a92b-b530-41fe-96ee-0350290a197a" containerID="03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300" exitCode=0
Sep 11 14:26:45.487022 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.486847    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"
Sep 11 14:26:45.487022 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.486872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn" event={"ID":"7e66a92b-b530-41fe-96ee-0350290a197a","Type":"ContainerDied","Data":"03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300"}
Sep 11 14:26:45.487022 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.486958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn" event={"ID":"7e66a92b-b530-41fe-96ee-0350290a197a","Type":"ContainerDied","Data":"2ee01f3e1683719e737e525b53c0727ac0dfec5c5183459af63535984d6cdd3b"}
Sep 11 14:26:45.487022 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.486981    2564 scope.go:117] "RemoveContainer" containerID="03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300"
Sep 11 14:26:45.488297 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.488270    2564 generic.go:356] "Generic (PLEG): container finished" podID="a7e206dd-7321-47a2-870e-f7090578f96a" containerID="b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19" exitCode=0
Sep 11 14:26:45.488415 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.488299    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx" event={"ID":"a7e206dd-7321-47a2-870e-f7090578f96a","Type":"ContainerDied","Data":"b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19"}
Sep 11 14:26:45.488415 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.488320    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"
Sep 11 14:26:45.488415 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.488347    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx" event={"ID":"a7e206dd-7321-47a2-870e-f7090578f96a","Type":"ContainerDied","Data":"3f032feb44c0d5e309606150e93b228a734232417f593394206a3b8530d2ec63"}
Sep 11 14:26:45.496983 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.496952    2564 scope.go:117] "RemoveContainer" containerID="03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300"
Sep 11 14:26:45.497292 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:26:45.497273    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300\": container with ID starting with 03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300 not found: ID does not exist" containerID="03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300"
Sep 11 14:26:45.497372 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.497300    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300"} err="failed to get container status \"03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300\": rpc error: code = NotFound desc = could not find container \"03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300\": container with ID starting with 03ce150aa96e6dadf1364a15d97a9cdfbfa23d7615516a1f3e837e78cade4300 not found: ID does not exist"
Sep 11 14:26:45.497372 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.497316    2564 scope.go:117] "RemoveContainer" containerID="b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19"
Sep 11 14:26:45.505010 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.504994    2564 scope.go:117] "RemoveContainer" containerID="b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19"
Sep 11 14:26:45.505270 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:26:45.505253    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19\": container with ID starting with b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19 not found: ID does not exist" containerID="b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19"
Sep 11 14:26:45.505330 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.505282    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19"} err="failed to get container status \"b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19\": rpc error: code = NotFound desc = could not find container \"b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19\": container with ID starting with b05b07da646c32e967313f2b4c655adee4bf4b5b928dc234f1a4ea5a180f1e19 not found: ID does not exist"
Sep 11 14:26:45.518580 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.518557    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"]
Sep 11 14:26:45.527120 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.527099    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-335a68-6f777bc59d-tnndn"]
Sep 11 14:26:45.541437 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.541414    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"]
Sep 11 14:26:45.546141 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:45.546120    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-335a68-68759bc987-s2mbx"]
Sep 11 14:26:46.275652 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:46.275599    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e66a92b-b530-41fe-96ee-0350290a197a" path="/var/lib/kubelet/pods/7e66a92b-b530-41fe-96ee-0350290a197a/volumes"
Sep 11 14:26:46.276027 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:26:46.275996    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7e206dd-7321-47a2-870e-f7090578f96a" path="/var/lib/kubelet/pods/a7e206dd-7321-47a2-870e-f7090578f96a/volumes"
Sep 11 14:28:19.507830 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.507793    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"]
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508203    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4257913-14d8-4f52-b7ef-6b5384331c2c" containerName="maas-api"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508216    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4257913-14d8-4f52-b7ef-6b5384331c2c" containerName="maas-api"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508230    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8734bbaa-bc41-4721-8e5b-ee73aad405f0" containerName="payload-pre-processing"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508236    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8734bbaa-bc41-4721-8e5b-ee73aad405f0" containerName="payload-pre-processing"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508243    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e206dd-7321-47a2-870e-f7090578f96a" containerName="payload-processing"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508249    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e206dd-7321-47a2-870e-f7090578f96a" containerName="payload-processing"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508260    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4e68cc5-2e69-435c-abd6-4f36652d9b19" containerName="maas-api"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508265    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4e68cc5-2e69-435c-abd6-4f36652d9b19" containerName="maas-api"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508275    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e66a92b-b530-41fe-96ee-0350290a197a" containerName="maas-api"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508280    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e66a92b-b530-41fe-96ee-0350290a197a" containerName="maas-api"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508291    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9966e7e9-6310-49de-9dbc-c25df5cdafe2" containerName="payload-processing"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508297    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9966e7e9-6310-49de-9dbc-c25df5cdafe2" containerName="payload-processing"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508371    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7e206dd-7321-47a2-870e-f7090578f96a" containerName="payload-processing"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508384    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8734bbaa-bc41-4721-8e5b-ee73aad405f0" containerName="payload-pre-processing"
Sep 11 14:28:19.508389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508398    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9966e7e9-6310-49de-9dbc-c25df5cdafe2" containerName="payload-processing"
Sep 11 14:28:19.509171 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508407    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e66a92b-b530-41fe-96ee-0350290a197a" containerName="maas-api"
Sep 11 14:28:19.509171 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508417    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4e68cc5-2e69-435c-abd6-4f36652d9b19" containerName="maas-api"
Sep 11 14:28:19.509171 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.508425    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4257913-14d8-4f52-b7ef-6b5384331c2c" containerName="maas-api"
Sep 11 14:28:19.511610 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.511587    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.515162 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.515136    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-30989f-serving-cert\""
Sep 11 14:28:19.532471 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.532444    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"]
Sep 11 14:28:19.657875 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.657839    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45n8f\" (UniqueName: \"kubernetes.io/projected/b858665b-a2b1-44af-af7f-70d0f4885f02-kube-api-access-45n8f\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.658057 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.657881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b858665b-a2b1-44af-af7f-70d0f4885f02-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.658057 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.657907    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-maas-api-tls\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.658057 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.657926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-metrics-certs\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.758838 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.758760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-maas-api-tls\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.758838 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.758799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-metrics-certs\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.759011 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.758912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45n8f\" (UniqueName: \"kubernetes.io/projected/b858665b-a2b1-44af-af7f-70d0f4885f02-kube-api-access-45n8f\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.759011 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.758952    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b858665b-a2b1-44af-af7f-70d0f4885f02-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.759560 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.759535    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b858665b-a2b1-44af-af7f-70d0f4885f02-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.761330 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.761308    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-metrics-certs\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.761428 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.761350    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-maas-api-tls\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.767400 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.767382    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45n8f\" (UniqueName: \"kubernetes.io/projected/b858665b-a2b1-44af-af7f-70d0f4885f02-kube-api-access-45n8f\") pod \"maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:19.822416 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:19.822393    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:20.108783 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:28:20.108752    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb858665b_a2b1_44af_af7f_70d0f4885f02.slice/crio-eea8e638e82769628ecde86631cda5dfb09368531ed25461a51788a05de21041 WatchSource:0}: Error finding container eea8e638e82769628ecde86631cda5dfb09368531ed25461a51788a05de21041: Status 404 returned error can't find the container with id eea8e638e82769628ecde86631cda5dfb09368531ed25461a51788a05de21041
Sep 11 14:28:20.110199 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.110175    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:28:20.110934 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.110901    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"]
Sep 11 14:28:20.184349 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.184311    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"]
Sep 11 14:28:20.189007 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.188985    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:20.189128 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.188984    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"]
Sep 11 14:28:20.191709 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.191686    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-30989f\""
Sep 11 14:28:20.192402 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.192387    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:28:20.201205 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.201180    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"]
Sep 11 14:28:20.210005 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.209982    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"]
Sep 11 14:28:20.263742 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.263707    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlr7d\" (UniqueName: \"kubernetes.io/projected/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-kube-api-access-mlr7d\") pod \"payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph\" (UID: \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:20.263889 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.263811    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph\" (UID: \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:20.364806 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.364724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph\" (UID: \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:20.364963 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.364828    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fc380c-d7af-44c1-92aa-9eac6f70110f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw\" (UID: \"78fc380c-d7af-44c1-92aa-9eac6f70110f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:28:20.364963 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.364903    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlr7d\" (UniqueName: \"kubernetes.io/projected/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-kube-api-access-mlr7d\") pod \"payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph\" (UID: \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:20.365103 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.365085    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcbx9\" (UniqueName: \"kubernetes.io/projected/78fc380c-d7af-44c1-92aa-9eac6f70110f-kube-api-access-xcbx9\") pod \"payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw\" (UID: \"78fc380c-d7af-44c1-92aa-9eac6f70110f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:28:20.365585 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.365564    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph\" (UID: \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:20.374306 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.374281    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlr7d\" (UniqueName: \"kubernetes.io/projected/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-kube-api-access-mlr7d\") pod \"payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph\" (UID: \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:20.466450 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.466413    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcbx9\" (UniqueName: \"kubernetes.io/projected/78fc380c-d7af-44c1-92aa-9eac6f70110f-kube-api-access-xcbx9\") pod \"payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw\" (UID: \"78fc380c-d7af-44c1-92aa-9eac6f70110f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:28:20.466619 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.466502    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fc380c-d7af-44c1-92aa-9eac6f70110f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw\" (UID: \"78fc380c-d7af-44c1-92aa-9eac6f70110f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:28:20.467232 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.467214    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fc380c-d7af-44c1-92aa-9eac6f70110f-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw\" (UID: \"78fc380c-d7af-44c1-92aa-9eac6f70110f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:28:20.476530 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.476511    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcbx9\" (UniqueName: \"kubernetes.io/projected/78fc380c-d7af-44c1-92aa-9eac6f70110f-kube-api-access-xcbx9\") pod \"payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw\" (UID: \"78fc380c-d7af-44c1-92aa-9eac6f70110f\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:28:20.501963 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.501935    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:20.510758 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.510735    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:28:20.779106 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.779074    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"]
Sep 11 14:28:20.781939 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:28:20.781904    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd68485f_35c7_4ab1_9b16_25bb88d7afe4.slice/crio-8cf2408eecc1207eff2ef5d150aa0fb21c55e86e83a2ee4939f7320be78df785 WatchSource:0}: Error finding container 8cf2408eecc1207eff2ef5d150aa0fb21c55e86e83a2ee4939f7320be78df785: Status 404 returned error can't find the container with id 8cf2408eecc1207eff2ef5d150aa0fb21c55e86e83a2ee4939f7320be78df785
Sep 11 14:28:20.788826 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.788800    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"]
Sep 11 14:28:20.795216 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:28:20.795190    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78fc380c_d7af_44c1_92aa_9eac6f70110f.slice/crio-224f4664a2b3e56d8faef8d02cb644d3ff7fa1b46f561aa4b59527af2b737cbf WatchSource:0}: Error finding container 224f4664a2b3e56d8faef8d02cb644d3ff7fa1b46f561aa4b59527af2b737cbf: Status 404 returned error can't find the container with id 224f4664a2b3e56d8faef8d02cb644d3ff7fa1b46f561aa4b59527af2b737cbf
Sep 11 14:28:20.959949 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.959843    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph" event={"ID":"fd68485f-35c7-4ab1-9b16-25bb88d7afe4","Type":"ContainerStarted","Data":"0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78"}
Sep 11 14:28:20.959949 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.959915    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph" event={"ID":"fd68485f-35c7-4ab1-9b16-25bb88d7afe4","Type":"ContainerStarted","Data":"8cf2408eecc1207eff2ef5d150aa0fb21c55e86e83a2ee4939f7320be78df785"}
Sep 11 14:28:20.960182 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.960036    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:20.961728 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.961696    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw" event={"ID":"78fc380c-d7af-44c1-92aa-9eac6f70110f","Type":"ContainerStarted","Data":"9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633"}
Sep 11 14:28:20.961864 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.961746    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw" event={"ID":"78fc380c-d7af-44c1-92aa-9eac6f70110f","Type":"ContainerStarted","Data":"224f4664a2b3e56d8faef8d02cb644d3ff7fa1b46f561aa4b59527af2b737cbf"}
Sep 11 14:28:20.961864 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.961773    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:28:20.963450 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.963412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6" event={"ID":"b858665b-a2b1-44af-af7f-70d0f4885f02","Type":"ContainerStarted","Data":"af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01"}
Sep 11 14:28:20.963587 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.963461    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:20.963587 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.963478    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6" event={"ID":"b858665b-a2b1-44af-af7f-70d0f4885f02","Type":"ContainerStarted","Data":"eea8e638e82769628ecde86631cda5dfb09368531ed25461a51788a05de21041"}
Sep 11 14:28:20.978034 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.977982    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph" podStartSLOduration=0.977965045 podStartE2EDuration="977.965045ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:28:20.976873487 +0000 UTC m=+1851.314449046" watchObservedRunningTime="2026-09-11 14:28:20.977965045 +0000 UTC m=+1851.315540599"
Sep 11 14:28:20.996181 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:20.996141    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw" podStartSLOduration=0.996126588 podStartE2EDuration="996.126588ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:28:20.994725008 +0000 UTC m=+1851.332300579" watchObservedRunningTime="2026-09-11 14:28:20.996126588 +0000 UTC m=+1851.333702149"
Sep 11 14:28:21.014635 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:21.014581    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6" podStartSLOduration=1.522068794 podStartE2EDuration="2.014567923s" totalImagesPullingTime="492.499129ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:28:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:28:21.012213244 +0000 UTC m=+1851.349788816" watchObservedRunningTime="2026-09-11 14:28:21.014567923 +0000 UTC m=+1851.352143482"
Sep 11 14:28:26.974167 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:26.974134    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:28:31.971046 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:31.971016    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:28:31.971498 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:28:31.971237    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:29:58.013316 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:29:58.013281    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw_78fc380c-d7af-44c1-92aa-9eac6f70110f/payload-processing/0.log"
Sep 11 14:30:07.685043 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.685010    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"]
Sep 11 14:30:07.688904 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.688881    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.695666 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.695642    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-e83e82-serving-cert\""
Sep 11 14:30:07.710218 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.710192    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqr82\" (UniqueName: \"kubernetes.io/projected/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-kube-api-access-sqr82\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.710350 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.710262    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.710350 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.710310    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.710350 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.710340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.713147 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.713123    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"]
Sep 11 14:30:07.812083 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.811848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.812083 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.811962    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.812083 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.812082    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqr82\" (UniqueName: \"kubernetes.io/projected/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-kube-api-access-sqr82\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.812385 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.812141    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.812385 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:30:07.812286    2564 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-serving-cert: secret "maas-api-e2e-ipp-cleanup-e83e82-serving-cert" not found
Sep 11 14:30:07.812385 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:30:07.812352    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-maas-api-tls podName:852fc4a4-7bad-4c3e-bc1a-6b2f89e19525 nodeName:}" failed. No retries permitted until 2026-09-11 14:30:08.312330976 +0000 UTC m=+1958.649906526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29" (UID: "852fc4a4-7bad-4c3e-bc1a-6b2f89e19525") : secret "maas-api-e2e-ipp-cleanup-e83e82-serving-cert" not found
Sep 11 14:30:07.813027 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.813007    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.815099 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.815078    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:07.829027 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:07.828978    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqr82\" (UniqueName: \"kubernetes.io/projected/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-kube-api-access-sqr82\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:08.316905 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.316866    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:08.319246 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.319215    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:08.599163 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.599078    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:08.696646 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.696595    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"]
Sep 11 14:30:08.701454 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.701433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:08.710031 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.710008    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-e83e82\""
Sep 11 14:30:08.710231 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.710214    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-e83e82-dockercfg-zhv9f\""
Sep 11 14:30:08.718352 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.718330    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"]
Sep 11 14:30:08.719667 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.719648    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82b62d5-9424-415b-97e3-1c984a936a6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs\" (UID: \"a82b62d5-9424-415b-97e3-1c984a936a6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:08.719785 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.719733    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5btwj\" (UniqueName: \"kubernetes.io/projected/a82b62d5-9424-415b-97e3-1c984a936a6f-kube-api-access-5btwj\") pod \"payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs\" (UID: \"a82b62d5-9424-415b-97e3-1c984a936a6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:08.737358 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.737330    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"]
Sep 11 14:30:08.740809 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.740794    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:08.755029 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.755003    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"]
Sep 11 14:30:08.821021 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.820987    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2956\" (UniqueName: \"kubernetes.io/projected/e6a0ead7-3c0b-4928-beb5-199d8f698f77-kube-api-access-t2956\") pod \"payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt\" (UID: \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:08.821183 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.821042    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82b62d5-9424-415b-97e3-1c984a936a6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs\" (UID: \"a82b62d5-9424-415b-97e3-1c984a936a6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:08.821183 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.821104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0ead7-3c0b-4928-beb5-199d8f698f77-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt\" (UID: \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:08.821183 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.821133    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5btwj\" (UniqueName: \"kubernetes.io/projected/a82b62d5-9424-415b-97e3-1c984a936a6f-kube-api-access-5btwj\") pod \"payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs\" (UID: \"a82b62d5-9424-415b-97e3-1c984a936a6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:08.821818 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.821795    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82b62d5-9424-415b-97e3-1c984a936a6f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs\" (UID: \"a82b62d5-9424-415b-97e3-1c984a936a6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:08.834264 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.834241    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5btwj\" (UniqueName: \"kubernetes.io/projected/a82b62d5-9424-415b-97e3-1c984a936a6f-kube-api-access-5btwj\") pod \"payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs\" (UID: \"a82b62d5-9424-415b-97e3-1c984a936a6f\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:08.883984 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.883961    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"]
Sep 11 14:30:08.886616 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:30:08.886584    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852fc4a4_7bad_4c3e_bc1a_6b2f89e19525.slice/crio-913e01642cf36a420433f89aa05e84c65916676702bd1f53d5c399365baba87a WatchSource:0}: Error finding container 913e01642cf36a420433f89aa05e84c65916676702bd1f53d5c399365baba87a: Status 404 returned error can't find the container with id 913e01642cf36a420433f89aa05e84c65916676702bd1f53d5c399365baba87a
Sep 11 14:30:08.922032 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.922006    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0ead7-3c0b-4928-beb5-199d8f698f77-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt\" (UID: \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:08.922153 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.922107    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2956\" (UniqueName: \"kubernetes.io/projected/e6a0ead7-3c0b-4928-beb5-199d8f698f77-kube-api-access-t2956\") pod \"payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt\" (UID: \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:08.922705 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.922687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0ead7-3c0b-4928-beb5-199d8f698f77-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt\" (UID: \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:08.931302 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:08.931285    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2956\" (UniqueName: \"kubernetes.io/projected/e6a0ead7-3c0b-4928-beb5-199d8f698f77-kube-api-access-t2956\") pod \"payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt\" (UID: \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:09.012595 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.012564    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:09.052154 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.052117    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:09.297618 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.297591    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"]
Sep 11 14:30:09.299617 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:30:09.299593    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82b62d5_9424_415b_97e3_1c984a936a6f.slice/crio-7248116bd33b14711453baf67abb4b1f50a017b5018e47ad758fb30feaf72108 WatchSource:0}: Error finding container 7248116bd33b14711453baf67abb4b1f50a017b5018e47ad758fb30feaf72108: Status 404 returned error can't find the container with id 7248116bd33b14711453baf67abb4b1f50a017b5018e47ad758fb30feaf72108
Sep 11 14:30:09.338538 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.338514    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"]
Sep 11 14:30:09.347691 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:30:09.347666    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a0ead7_3c0b_4928_beb5_199d8f698f77.slice/crio-c6ec6108cc0e957847921d91d940381d7881d5fd1126074ca025ee9d73a74faa WatchSource:0}: Error finding container c6ec6108cc0e957847921d91d940381d7881d5fd1126074ca025ee9d73a74faa: Status 404 returned error can't find the container with id c6ec6108cc0e957847921d91d940381d7881d5fd1126074ca025ee9d73a74faa
Sep 11 14:30:09.503464 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.503423    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs" event={"ID":"a82b62d5-9424-415b-97e3-1c984a936a6f","Type":"ContainerStarted","Data":"17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65"}
Sep 11 14:30:09.503658 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.503487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs" event={"ID":"a82b62d5-9424-415b-97e3-1c984a936a6f","Type":"ContainerStarted","Data":"7248116bd33b14711453baf67abb4b1f50a017b5018e47ad758fb30feaf72108"}
Sep 11 14:30:09.503658 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.503536    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:09.505284 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.505254    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt" event={"ID":"e6a0ead7-3c0b-4928-beb5-199d8f698f77","Type":"ContainerStarted","Data":"ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049"}
Sep 11 14:30:09.505406 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.505299    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt" event={"ID":"e6a0ead7-3c0b-4928-beb5-199d8f698f77","Type":"ContainerStarted","Data":"c6ec6108cc0e957847921d91d940381d7881d5fd1126074ca025ee9d73a74faa"}
Sep 11 14:30:09.505406 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.505326    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:09.506948 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.506919    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29" event={"ID":"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525","Type":"ContainerStarted","Data":"34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208"}
Sep 11 14:30:09.507040 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.506964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29" event={"ID":"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525","Type":"ContainerStarted","Data":"913e01642cf36a420433f89aa05e84c65916676702bd1f53d5c399365baba87a"}
Sep 11 14:30:09.507040 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.506985    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:09.523944 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.523906    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs" podStartSLOduration=1.523893976 podStartE2EDuration="1.523893976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:30:09.522457142 +0000 UTC m=+1959.860032714" watchObservedRunningTime="2026-09-11 14:30:09.523893976 +0000 UTC m=+1959.861469536"
Sep 11 14:30:09.547613 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.547532    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29" podStartSLOduration=2.108596578 podStartE2EDuration="2.547518445s" totalImagesPullingTime="438.921867ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:30:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:30:09.546576718 +0000 UTC m=+1959.884152281" watchObservedRunningTime="2026-09-11 14:30:09.547518445 +0000 UTC m=+1959.885094041"
Sep 11 14:30:09.570909 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:09.570869    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt" podStartSLOduration=1.570855889 podStartE2EDuration="1.570855889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:30:09.568079827 +0000 UTC m=+1959.905655401" watchObservedRunningTime="2026-09-11 14:30:09.570855889 +0000 UTC m=+1959.908431450"
Sep 11 14:30:15.517687 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:15.517654    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:20.513175 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:20.513145    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:20.513579 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:20.513372    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:30.412612 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:30.412541    2564 scope.go:117] "RemoveContainer" containerID="b1f3d73a13ce6c87999bb9b38d4d41bc289cae2d8b7c270f246476cf797954bb"
Sep 11 14:30:44.692503 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:44.692470    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"]
Sep 11 14:30:44.693095 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:44.692740    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29" podUID="852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" containerName="maas-api" containerID="cri-o://34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208" gracePeriod=30
Sep 11 14:30:44.770816 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:44.770785    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"]
Sep 11 14:30:44.771073 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:44.771046    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt" podUID="e6a0ead7-3c0b-4928-beb5-199d8f698f77" containerName="payload-processing" containerID="cri-o://ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049" gracePeriod=30
Sep 11 14:30:44.785556 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:44.785528    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"]
Sep 11 14:30:44.785823 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:44.785787    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs" podUID="a82b62d5-9424-415b-97e3-1c984a936a6f" containerName="payload-pre-processing" containerID="cri-o://17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65" gracePeriod=30
Sep 11 14:30:45.039693 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.039649    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:45.077235 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.077208    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:45.085936 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.085914    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:45.157634 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.157597    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-openshift-service-ca\") pod \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") "
Sep 11 14:30:45.157795 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.157684    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-maas-api-tls\") pod \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") "
Sep 11 14:30:45.157795 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.157719    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-metrics-certs\") pod \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") "
Sep 11 14:30:45.157913 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.157806    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqr82\" (UniqueName: \"kubernetes.io/projected/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-kube-api-access-sqr82\") pod \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\" (UID: \"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525\") "
Sep 11 14:30:45.158023 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.157995    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" (UID: "852fc4a4-7bad-4c3e-bc1a-6b2f89e19525"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:30:45.158168 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.158149    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:30:45.159960 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.159936    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" (UID: "852fc4a4-7bad-4c3e-bc1a-6b2f89e19525"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:30:45.160065 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.159988    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" (UID: "852fc4a4-7bad-4c3e-bc1a-6b2f89e19525"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:30:45.160398 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.160383    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-kube-api-access-sqr82" (OuterVolumeSpecName: "kube-api-access-sqr82") pod "852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" (UID: "852fc4a4-7bad-4c3e-bc1a-6b2f89e19525"). InnerVolumeSpecName "kube-api-access-sqr82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:30:45.258513 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.258434    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82b62d5-9424-415b-97e3-1c984a936a6f-plugins-config-volume\") pod \"a82b62d5-9424-415b-97e3-1c984a936a6f\" (UID: \"a82b62d5-9424-415b-97e3-1c984a936a6f\") "
Sep 11 14:30:45.258513 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.258493    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5btwj\" (UniqueName: \"kubernetes.io/projected/a82b62d5-9424-415b-97e3-1c984a936a6f-kube-api-access-5btwj\") pod \"a82b62d5-9424-415b-97e3-1c984a936a6f\" (UID: \"a82b62d5-9424-415b-97e3-1c984a936a6f\") "
Sep 11 14:30:45.258790 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.258580    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2956\" (UniqueName: \"kubernetes.io/projected/e6a0ead7-3c0b-4928-beb5-199d8f698f77-kube-api-access-t2956\") pod \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\" (UID: \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\") "
Sep 11 14:30:45.258790 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.258647    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0ead7-3c0b-4928-beb5-199d8f698f77-plugins-config-volume\") pod \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\" (UID: \"e6a0ead7-3c0b-4928-beb5-199d8f698f77\") "
Sep 11 14:30:45.258973 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.258955    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqr82\" (UniqueName: \"kubernetes.io/projected/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-kube-api-access-sqr82\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:30:45.259036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.258986    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:30:45.259036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.259000    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:30:45.259036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.258951    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a82b62d5-9424-415b-97e3-1c984a936a6f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a82b62d5-9424-415b-97e3-1c984a936a6f" (UID: "a82b62d5-9424-415b-97e3-1c984a936a6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:30:45.259171 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.259101    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6a0ead7-3c0b-4928-beb5-199d8f698f77-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e6a0ead7-3c0b-4928-beb5-199d8f698f77" (UID: "e6a0ead7-3c0b-4928-beb5-199d8f698f77"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:30:45.261077 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.261052    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a82b62d5-9424-415b-97e3-1c984a936a6f-kube-api-access-5btwj" (OuterVolumeSpecName: "kube-api-access-5btwj") pod "a82b62d5-9424-415b-97e3-1c984a936a6f" (UID: "a82b62d5-9424-415b-97e3-1c984a936a6f"). InnerVolumeSpecName "kube-api-access-5btwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:30:45.261177 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.261086    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6a0ead7-3c0b-4928-beb5-199d8f698f77-kube-api-access-t2956" (OuterVolumeSpecName: "kube-api-access-t2956") pod "e6a0ead7-3c0b-4928-beb5-199d8f698f77" (UID: "e6a0ead7-3c0b-4928-beb5-199d8f698f77"). InnerVolumeSpecName "kube-api-access-t2956". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:30:45.360091 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.360050    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2956\" (UniqueName: \"kubernetes.io/projected/e6a0ead7-3c0b-4928-beb5-199d8f698f77-kube-api-access-t2956\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:30:45.360091 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.360084    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e6a0ead7-3c0b-4928-beb5-199d8f698f77-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:30:45.360091 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.360097    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a82b62d5-9424-415b-97e3-1c984a936a6f-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:30:45.360318 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.360109    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5btwj\" (UniqueName: \"kubernetes.io/projected/a82b62d5-9424-415b-97e3-1c984a936a6f-kube-api-access-5btwj\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:30:45.698266 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.698229    2564 generic.go:356] "Generic (PLEG): container finished" podID="e6a0ead7-3c0b-4928-beb5-199d8f698f77" containerID="ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049" exitCode=0
Sep 11 14:30:45.698709 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.698273    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"
Sep 11 14:30:45.698709 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.698320    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt" event={"ID":"e6a0ead7-3c0b-4928-beb5-199d8f698f77","Type":"ContainerDied","Data":"ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049"}
Sep 11 14:30:45.698709 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.698367    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt" event={"ID":"e6a0ead7-3c0b-4928-beb5-199d8f698f77","Type":"ContainerDied","Data":"c6ec6108cc0e957847921d91d940381d7881d5fd1126074ca025ee9d73a74faa"}
Sep 11 14:30:45.698709 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.698388    2564 scope.go:117] "RemoveContainer" containerID="ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049"
Sep 11 14:30:45.699753 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.699733    2564 generic.go:356] "Generic (PLEG): container finished" podID="852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" containerID="34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208" exitCode=0
Sep 11 14:30:45.699817 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.699785    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"
Sep 11 14:30:45.699871 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.699828    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29" event={"ID":"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525","Type":"ContainerDied","Data":"34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208"}
Sep 11 14:30:45.699931 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.699875    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29" event={"ID":"852fc4a4-7bad-4c3e-bc1a-6b2f89e19525","Type":"ContainerDied","Data":"913e01642cf36a420433f89aa05e84c65916676702bd1f53d5c399365baba87a"}
Sep 11 14:30:45.701445 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.701408    2564 generic.go:356] "Generic (PLEG): container finished" podID="a82b62d5-9424-415b-97e3-1c984a936a6f" containerID="17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65" exitCode=0
Sep 11 14:30:45.701560 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.701459    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"
Sep 11 14:30:45.701560 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.701455    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs" event={"ID":"a82b62d5-9424-415b-97e3-1c984a936a6f","Type":"ContainerDied","Data":"17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65"}
Sep 11 14:30:45.701720 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.701593    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs" event={"ID":"a82b62d5-9424-415b-97e3-1c984a936a6f","Type":"ContainerDied","Data":"7248116bd33b14711453baf67abb4b1f50a017b5018e47ad758fb30feaf72108"}
Sep 11 14:30:45.721410 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.721390    2564 scope.go:117] "RemoveContainer" containerID="ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049"
Sep 11 14:30:45.721867 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:30:45.721846    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049\": container with ID starting with ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049 not found: ID does not exist" containerID="ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049"
Sep 11 14:30:45.721967 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.721878    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049"} err="failed to get container status \"ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049\": rpc error: code = NotFound desc = could not find container \"ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049\": container with ID starting with ef025ce35949ba9277cb76a4f982952d4df03c2d876e34ff24536ed556358049 not found: ID does not exist"
Sep 11 14:30:45.721967 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.721901    2564 scope.go:117] "RemoveContainer" containerID="34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208"
Sep 11 14:30:45.729932 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.729916    2564 scope.go:117] "RemoveContainer" containerID="34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208"
Sep 11 14:30:45.730228 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:30:45.730209    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208\": container with ID starting with 34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208 not found: ID does not exist" containerID="34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208"
Sep 11 14:30:45.730294 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.730233    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208"} err="failed to get container status \"34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208\": rpc error: code = NotFound desc = could not find container \"34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208\": container with ID starting with 34b7466165a3bc7026ff21426151992f0a455daab284c240f90ec40cd04cc208 not found: ID does not exist"
Sep 11 14:30:45.730294 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.730249    2564 scope.go:117] "RemoveContainer" containerID="17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65"
Sep 11 14:30:45.738291 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.738270    2564 scope.go:117] "RemoveContainer" containerID="17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65"
Sep 11 14:30:45.738547 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:30:45.738526    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65\": container with ID starting with 17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65 not found: ID does not exist" containerID="17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65"
Sep 11 14:30:45.738617 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.738554    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65"} err="failed to get container status \"17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65\": rpc error: code = NotFound desc = could not find container \"17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65\": container with ID starting with 17e7bfb7f8b790182943c36239d0a9ac4f7294a2a08f557c123610d50b4b9d65 not found: ID does not exist"
Sep 11 14:30:45.740296 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.740278    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"]
Sep 11 14:30:45.746322 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.746303    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-e83e82-574cf88bf4-bz6fs"]
Sep 11 14:30:45.762861 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.762837    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"]
Sep 11 14:30:45.771880 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.771859    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-e83e82-5f789c8b55-nss29"]
Sep 11 14:30:45.785065 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.785045    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"]
Sep 11 14:30:45.790186 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:45.790164    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-e83e82-67f9d6c86f-g7gtt"]
Sep 11 14:30:46.276186 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:46.276155    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" path="/var/lib/kubelet/pods/852fc4a4-7bad-4c3e-bc1a-6b2f89e19525/volumes"
Sep 11 14:30:46.276515 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:46.276502    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a82b62d5-9424-415b-97e3-1c984a936a6f" path="/var/lib/kubelet/pods/a82b62d5-9424-415b-97e3-1c984a936a6f/volumes"
Sep 11 14:30:46.276868 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:30:46.276854    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6a0ead7-3c0b-4928-beb5-199d8f698f77" path="/var/lib/kubelet/pods/e6a0ead7-3c0b-4928-beb5-199d8f698f77/volumes"
Sep 11 14:31:11.907646 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:11.907598    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"]
Sep 11 14:31:11.908104 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:11.907914    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6" podUID="b858665b-a2b1-44af-af7f-70d0f4885f02" containerName="maas-api" containerID="cri-o://af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01" gracePeriod=30
Sep 11 14:31:11.969831 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:11.969764    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6" podUID="b858665b-a2b1-44af-af7f-70d0f4885f02" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.61:8443/health\": dial tcp 10.132.0.61:8443: connect: connection refused"
Sep 11 14:31:12.031354 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.031321    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"]
Sep 11 14:31:12.031574 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.031548    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw" podUID="78fc380c-d7af-44c1-92aa-9eac6f70110f" containerName="payload-processing" containerID="cri-o://9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633" gracePeriod=30
Sep 11 14:31:12.051618 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.051589    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"]
Sep 11 14:31:12.051888 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.051860    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph" podUID="fd68485f-35c7-4ab1-9b16-25bb88d7afe4" containerName="payload-pre-processing" containerID="cri-o://0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78" gracePeriod=30
Sep 11 14:31:12.178718 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.178690    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:31:12.300067 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.300037    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b858665b-a2b1-44af-af7f-70d0f4885f02-openshift-service-ca\") pod \"b858665b-a2b1-44af-af7f-70d0f4885f02\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") "
Sep 11 14:31:12.300268 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.300101    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-metrics-certs\") pod \"b858665b-a2b1-44af-af7f-70d0f4885f02\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") "
Sep 11 14:31:12.300268 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.300134    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-maas-api-tls\") pod \"b858665b-a2b1-44af-af7f-70d0f4885f02\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") "
Sep 11 14:31:12.300268 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.300194    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45n8f\" (UniqueName: \"kubernetes.io/projected/b858665b-a2b1-44af-af7f-70d0f4885f02-kube-api-access-45n8f\") pod \"b858665b-a2b1-44af-af7f-70d0f4885f02\" (UID: \"b858665b-a2b1-44af-af7f-70d0f4885f02\") "
Sep 11 14:31:12.300464 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.300436    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b858665b-a2b1-44af-af7f-70d0f4885f02-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b858665b-a2b1-44af-af7f-70d0f4885f02" (UID: "b858665b-a2b1-44af-af7f-70d0f4885f02"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:31:12.303718 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.303687    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b858665b-a2b1-44af-af7f-70d0f4885f02" (UID: "b858665b-a2b1-44af-af7f-70d0f4885f02"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:31:12.303835 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.303745    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b858665b-a2b1-44af-af7f-70d0f4885f02" (UID: "b858665b-a2b1-44af-af7f-70d0f4885f02"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:31:12.303901 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.303881    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b858665b-a2b1-44af-af7f-70d0f4885f02-kube-api-access-45n8f" (OuterVolumeSpecName: "kube-api-access-45n8f") pod "b858665b-a2b1-44af-af7f-70d0f4885f02" (UID: "b858665b-a2b1-44af-af7f-70d0f4885f02"). InnerVolumeSpecName "kube-api-access-45n8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:31:12.320588 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.320566    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:31:12.324036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.324016    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:31:12.401414 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.401381    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-plugins-config-volume\") pod \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\" (UID: \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\") "
Sep 11 14:31:12.401604 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.401460    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fc380c-d7af-44c1-92aa-9eac6f70110f-plugins-config-volume\") pod \"78fc380c-d7af-44c1-92aa-9eac6f70110f\" (UID: \"78fc380c-d7af-44c1-92aa-9eac6f70110f\") "
Sep 11 14:31:12.401604 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.401560    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlr7d\" (UniqueName: \"kubernetes.io/projected/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-kube-api-access-mlr7d\") pod \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\" (UID: \"fd68485f-35c7-4ab1-9b16-25bb88d7afe4\") "
Sep 11 14:31:12.401604 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.401589    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcbx9\" (UniqueName: \"kubernetes.io/projected/78fc380c-d7af-44c1-92aa-9eac6f70110f-kube-api-access-xcbx9\") pod \"78fc380c-d7af-44c1-92aa-9eac6f70110f\" (UID: \"78fc380c-d7af-44c1-92aa-9eac6f70110f\") "
Sep 11 14:31:12.401811 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.401768    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd68485f-35c7-4ab1-9b16-25bb88d7afe4" (UID: "fd68485f-35c7-4ab1-9b16-25bb88d7afe4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:31:12.401877 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.401841    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78fc380c-d7af-44c1-92aa-9eac6f70110f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78fc380c-d7af-44c1-92aa-9eac6f70110f" (UID: "78fc380c-d7af-44c1-92aa-9eac6f70110f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:31:12.402152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.402133    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45n8f\" (UniqueName: \"kubernetes.io/projected/b858665b-a2b1-44af-af7f-70d0f4885f02-kube-api-access-45n8f\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:31:12.402243 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.402155    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:31:12.402243 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.402168    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78fc380c-d7af-44c1-92aa-9eac6f70110f-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:31:12.402243 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.402183    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b858665b-a2b1-44af-af7f-70d0f4885f02-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:31:12.402243 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.402197    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:31:12.402243 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.402211    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b858665b-a2b1-44af-af7f-70d0f4885f02-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:31:12.404883 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.404852    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78fc380c-d7af-44c1-92aa-9eac6f70110f-kube-api-access-xcbx9" (OuterVolumeSpecName: "kube-api-access-xcbx9") pod "78fc380c-d7af-44c1-92aa-9eac6f70110f" (UID: "78fc380c-d7af-44c1-92aa-9eac6f70110f"). InnerVolumeSpecName "kube-api-access-xcbx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:31:12.404997 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.404881    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-kube-api-access-mlr7d" (OuterVolumeSpecName: "kube-api-access-mlr7d") pod "fd68485f-35c7-4ab1-9b16-25bb88d7afe4" (UID: "fd68485f-35c7-4ab1-9b16-25bb88d7afe4"). InnerVolumeSpecName "kube-api-access-mlr7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:31:12.502985 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.502903    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlr7d\" (UniqueName: \"kubernetes.io/projected/fd68485f-35c7-4ab1-9b16-25bb88d7afe4-kube-api-access-mlr7d\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:31:12.502985 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.502929    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcbx9\" (UniqueName: \"kubernetes.io/projected/78fc380c-d7af-44c1-92aa-9eac6f70110f-kube-api-access-xcbx9\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:31:12.845675 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.845641    2564 generic.go:356] "Generic (PLEG): container finished" podID="fd68485f-35c7-4ab1-9b16-25bb88d7afe4" containerID="0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78" exitCode=0
Sep 11 14:31:12.845829 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.845717    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"
Sep 11 14:31:12.845829 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.845716    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph" event={"ID":"fd68485f-35c7-4ab1-9b16-25bb88d7afe4","Type":"ContainerDied","Data":"0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78"}
Sep 11 14:31:12.845829 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.845803    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph" event={"ID":"fd68485f-35c7-4ab1-9b16-25bb88d7afe4","Type":"ContainerDied","Data":"8cf2408eecc1207eff2ef5d150aa0fb21c55e86e83a2ee4939f7320be78df785"}
Sep 11 14:31:12.846010 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.845834    2564 scope.go:117] "RemoveContainer" containerID="0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78"
Sep 11 14:31:12.847190 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.847165    2564 generic.go:356] "Generic (PLEG): container finished" podID="78fc380c-d7af-44c1-92aa-9eac6f70110f" containerID="9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633" exitCode=0
Sep 11 14:31:12.847311 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.847220    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"
Sep 11 14:31:12.847311 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.847235    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw" event={"ID":"78fc380c-d7af-44c1-92aa-9eac6f70110f","Type":"ContainerDied","Data":"9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633"}
Sep 11 14:31:12.847311 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.847280    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw" event={"ID":"78fc380c-d7af-44c1-92aa-9eac6f70110f","Type":"ContainerDied","Data":"224f4664a2b3e56d8faef8d02cb644d3ff7fa1b46f561aa4b59527af2b737cbf"}
Sep 11 14:31:12.849011 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.848986    2564 generic.go:356] "Generic (PLEG): container finished" podID="b858665b-a2b1-44af-af7f-70d0f4885f02" containerID="af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01" exitCode=0
Sep 11 14:31:12.849119 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.849024    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6" event={"ID":"b858665b-a2b1-44af-af7f-70d0f4885f02","Type":"ContainerDied","Data":"af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01"}
Sep 11 14:31:12.849119 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.849063    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6" event={"ID":"b858665b-a2b1-44af-af7f-70d0f4885f02","Type":"ContainerDied","Data":"eea8e638e82769628ecde86631cda5dfb09368531ed25461a51788a05de21041"}
Sep 11 14:31:12.849119 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.849112    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"
Sep 11 14:31:12.860326 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.860297    2564 scope.go:117] "RemoveContainer" containerID="0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78"
Sep 11 14:31:12.861267 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:31:12.861088    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78\": container with ID starting with 0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78 not found: ID does not exist" containerID="0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78"
Sep 11 14:31:12.861267 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.861128    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78"} err="failed to get container status \"0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78\": rpc error: code = NotFound desc = could not find container \"0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78\": container with ID starting with 0b409ecce6026d44892a2f9dd42a69c6c4c5d634db733a076bb65ef330071d78 not found: ID does not exist"
Sep 11 14:31:12.861267 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.861153    2564 scope.go:117] "RemoveContainer" containerID="9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633"
Sep 11 14:31:12.870120 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.870094    2564 scope.go:117] "RemoveContainer" containerID="9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633"
Sep 11 14:31:12.870386 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:31:12.870368    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633\": container with ID starting with 9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633 not found: ID does not exist" containerID="9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633"
Sep 11 14:31:12.870451 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.870391    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633"} err="failed to get container status \"9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633\": rpc error: code = NotFound desc = could not find container \"9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633\": container with ID starting with 9242d9bc7be3073cbc4c5e6d873150f60d1c936a23a897ae1e41fd1d0d0c4633 not found: ID does not exist"
Sep 11 14:31:12.870451 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.870407    2564 scope.go:117] "RemoveContainer" containerID="af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01"
Sep 11 14:31:12.878334 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.878317    2564 scope.go:117] "RemoveContainer" containerID="af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01"
Sep 11 14:31:12.878617 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:31:12.878598    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01\": container with ID starting with af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01 not found: ID does not exist" containerID="af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01"
Sep 11 14:31:12.878695 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.878639    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01"} err="failed to get container status \"af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01\": rpc error: code = NotFound desc = could not find container \"af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01\": container with ID starting with af1d04702f3988d6e81b2d4c8b7d84ba46de537867f7647af0a6c581ef09bb01 not found: ID does not exist"
Sep 11 14:31:12.893884 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.893856    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"]
Sep 11 14:31:12.895603 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.895583    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-30989f-675db65d55-5whvw"]
Sep 11 14:31:12.917064 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.917029    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"]
Sep 11 14:31:12.921312 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.921289    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-30989f-77b6dcbd8b-4t8l6"]
Sep 11 14:31:12.937289 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.937264    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"]
Sep 11 14:31:12.944539 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:12.944513    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-30989f-55c6764894-qqlph"]
Sep 11 14:31:14.276383 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:14.276349    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78fc380c-d7af-44c1-92aa-9eac6f70110f" path="/var/lib/kubelet/pods/78fc380c-d7af-44c1-92aa-9eac6f70110f/volumes"
Sep 11 14:31:14.276974 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:14.276952    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b858665b-a2b1-44af-af7f-70d0f4885f02" path="/var/lib/kubelet/pods/b858665b-a2b1-44af-af7f-70d0f4885f02/volumes"
Sep 11 14:31:14.277365 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:31:14.277347    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd68485f-35c7-4ab1-9b16-25bb88d7afe4" path="/var/lib/kubelet/pods/fd68485f-35c7-4ab1-9b16-25bb88d7afe4/volumes"
Sep 11 14:32:31.374769 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.374739    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"]
Sep 11 14:32:31.375253 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.374936    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl" podUID="fef2b257-4e36-4d7d-a5a9-b6609f3a6957" containerName="maas-api" containerID="cri-o://9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d" gracePeriod=30
Sep 11 14:32:31.626214 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.626157    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:32:31.719967 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.719934    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-metrics-certs\") pod \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") "
Sep 11 14:32:31.719967 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.719970    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j7pb\" (UniqueName: \"kubernetes.io/projected/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-kube-api-access-5j7pb\") pod \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") "
Sep 11 14:32:31.720188 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.720023    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-openshift-service-ca\") pod \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") "
Sep 11 14:32:31.720188 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.720075    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-maas-api-tls\") pod \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\" (UID: \"fef2b257-4e36-4d7d-a5a9-b6609f3a6957\") "
Sep 11 14:32:31.720433 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.720408    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fef2b257-4e36-4d7d-a5a9-b6609f3a6957" (UID: "fef2b257-4e36-4d7d-a5a9-b6609f3a6957"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:32:31.722147 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.722123    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fef2b257-4e36-4d7d-a5a9-b6609f3a6957" (UID: "fef2b257-4e36-4d7d-a5a9-b6609f3a6957"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:32:31.722147 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.722141    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fef2b257-4e36-4d7d-a5a9-b6609f3a6957" (UID: "fef2b257-4e36-4d7d-a5a9-b6609f3a6957"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:32:31.722305 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.722174    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-kube-api-access-5j7pb" (OuterVolumeSpecName: "kube-api-access-5j7pb") pod "fef2b257-4e36-4d7d-a5a9-b6609f3a6957" (UID: "fef2b257-4e36-4d7d-a5a9-b6609f3a6957"). InnerVolumeSpecName "kube-api-access-5j7pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:32:31.821715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.821686    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:31.821715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.821712    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:31.821715 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.821722    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:31.821915 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:31.821730    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j7pb\" (UniqueName: \"kubernetes.io/projected/fef2b257-4e36-4d7d-a5a9-b6609f3a6957-kube-api-access-5j7pb\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:32.230313 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.230276    2564 generic.go:356] "Generic (PLEG): container finished" podID="fef2b257-4e36-4d7d-a5a9-b6609f3a6957" containerID="9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d" exitCode=0
Sep 11 14:32:32.230313 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.230313    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"
Sep 11 14:32:32.230530 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.230325    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl" event={"ID":"fef2b257-4e36-4d7d-a5a9-b6609f3a6957","Type":"ContainerDied","Data":"9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d"}
Sep 11 14:32:32.230530 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.230370    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl" event={"ID":"fef2b257-4e36-4d7d-a5a9-b6609f3a6957","Type":"ContainerDied","Data":"65d3809aaee62909da3aadb0b1009ee47c8d5fd142dd883c47af6b7370dfdbd7"}
Sep 11 14:32:32.230530 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.230392    2564 scope.go:117] "RemoveContainer" containerID="9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d"
Sep 11 14:32:32.239535 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.239520    2564 scope.go:117] "RemoveContainer" containerID="9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d"
Sep 11 14:32:32.239841 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:32:32.239821    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d\": container with ID starting with 9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d not found: ID does not exist" containerID="9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d"
Sep 11 14:32:32.239928 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.239849    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d"} err="failed to get container status \"9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d\": rpc error: code = NotFound desc = could not find container \"9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d\": container with ID starting with 9ab142195cce0bd8ab06e44a547ebbf5b20772b1c25f80a8c18a024c96a5b36d not found: ID does not exist"
Sep 11 14:32:32.254685 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.254653    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"]
Sep 11 14:32:32.261098 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.261074    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1e2e7d-5d6d77f68-2npbl"]
Sep 11 14:32:32.275095 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:32.275075    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fef2b257-4e36-4d7d-a5a9-b6609f3a6957" path="/var/lib/kubelet/pods/fef2b257-4e36-4d7d-a5a9-b6609f3a6957/volumes"
Sep 11 14:32:50.604699 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.604662    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"]
Sep 11 14:32:50.605184 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.604902    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2" podUID="81e8bf04-451a-40c0-a3e3-0827a1365196" containerName="maas-api" containerID="cri-o://5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1" gracePeriod=30
Sep 11 14:32:50.733111 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.733079    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"]
Sep 11 14:32:50.733335 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.733311    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9" podUID="130aaa7d-5fc3-40db-bbea-21b0a4beda7c" containerName="payload-pre-processing" containerID="cri-o://a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8" gracePeriod=30
Sep 11 14:32:50.862022 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.861964    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:32:50.991856 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.991828    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9cv6\" (UniqueName: \"kubernetes.io/projected/81e8bf04-451a-40c0-a3e3-0827a1365196-kube-api-access-v9cv6\") pod \"81e8bf04-451a-40c0-a3e3-0827a1365196\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") "
Sep 11 14:32:50.991993 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.991871    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-metrics-certs\") pod \"81e8bf04-451a-40c0-a3e3-0827a1365196\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") "
Sep 11 14:32:50.991993 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.991889    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e8bf04-451a-40c0-a3e3-0827a1365196-openshift-service-ca\") pod \"81e8bf04-451a-40c0-a3e3-0827a1365196\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") "
Sep 11 14:32:50.991993 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.991980    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls\") pod \"81e8bf04-451a-40c0-a3e3-0827a1365196\" (UID: \"81e8bf04-451a-40c0-a3e3-0827a1365196\") "
Sep 11 14:32:50.992299 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.992270    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81e8bf04-451a-40c0-a3e3-0827a1365196-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81e8bf04-451a-40c0-a3e3-0827a1365196" (UID: "81e8bf04-451a-40c0-a3e3-0827a1365196"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:32:50.994160 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.994136    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81e8bf04-451a-40c0-a3e3-0827a1365196-kube-api-access-v9cv6" (OuterVolumeSpecName: "kube-api-access-v9cv6") pod "81e8bf04-451a-40c0-a3e3-0827a1365196" (UID: "81e8bf04-451a-40c0-a3e3-0827a1365196"). InnerVolumeSpecName "kube-api-access-v9cv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:32:50.994280 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.994170    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81e8bf04-451a-40c0-a3e3-0827a1365196" (UID: "81e8bf04-451a-40c0-a3e3-0827a1365196"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:32:50.994341 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.994309    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "81e8bf04-451a-40c0-a3e3-0827a1365196" (UID: "81e8bf04-451a-40c0-a3e3-0827a1365196"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:32:50.994817 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:50.994800    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:32:51.093168 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.093138    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-plugins-config-volume\") pod \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\" (UID: \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\") "
Sep 11 14:32:51.093330 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.093176    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgqhb\" (UniqueName: \"kubernetes.io/projected/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-kube-api-access-dgqhb\") pod \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\" (UID: \"130aaa7d-5fc3-40db-bbea-21b0a4beda7c\") "
Sep 11 14:32:51.093539 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.093511    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "130aaa7d-5fc3-40db-bbea-21b0a4beda7c" (UID: "130aaa7d-5fc3-40db-bbea-21b0a4beda7c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:32:51.093654 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.093608    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:51.093654 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.093642    2564 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-plugins-config-volume\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:51.093772 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.093660    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9cv6\" (UniqueName: \"kubernetes.io/projected/81e8bf04-451a-40c0-a3e3-0827a1365196-kube-api-access-v9cv6\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:51.093772 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.093672    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e8bf04-451a-40c0-a3e3-0827a1365196-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:51.093772 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.093707    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e8bf04-451a-40c0-a3e3-0827a1365196-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:51.095403 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.095383    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-kube-api-access-dgqhb" (OuterVolumeSpecName: "kube-api-access-dgqhb") pod "130aaa7d-5fc3-40db-bbea-21b0a4beda7c" (UID: "130aaa7d-5fc3-40db-bbea-21b0a4beda7c"). InnerVolumeSpecName "kube-api-access-dgqhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:32:51.195017 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.194937    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgqhb\" (UniqueName: \"kubernetes.io/projected/130aaa7d-5fc3-40db-bbea-21b0a4beda7c-kube-api-access-dgqhb\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:32:51.318410 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.318377    2564 generic.go:356] "Generic (PLEG): container finished" podID="81e8bf04-451a-40c0-a3e3-0827a1365196" containerID="5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1" exitCode=0
Sep 11 14:32:51.318592 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.318428    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"
Sep 11 14:32:51.318592 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.318459    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2" event={"ID":"81e8bf04-451a-40c0-a3e3-0827a1365196","Type":"ContainerDied","Data":"5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1"}
Sep 11 14:32:51.318592 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.318515    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2" event={"ID":"81e8bf04-451a-40c0-a3e3-0827a1365196","Type":"ContainerDied","Data":"ffd29274114c8965cd16405a01fa269a21239dd3ef4823577fe8132c9beda168"}
Sep 11 14:32:51.318592 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.318548    2564 scope.go:117] "RemoveContainer" containerID="5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1"
Sep 11 14:32:51.319884 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.319866    2564 generic.go:356] "Generic (PLEG): container finished" podID="130aaa7d-5fc3-40db-bbea-21b0a4beda7c" containerID="a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8" exitCode=0
Sep 11 14:32:51.319983 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.319894    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9" event={"ID":"130aaa7d-5fc3-40db-bbea-21b0a4beda7c","Type":"ContainerDied","Data":"a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8"}
Sep 11 14:32:51.319983 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.319925    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9" event={"ID":"130aaa7d-5fc3-40db-bbea-21b0a4beda7c","Type":"ContainerDied","Data":"19bf9d9fdb3032e560c2b580a1c20b9583bbba43ddf4b3f6054426d787bab30b"}
Sep 11 14:32:51.319983 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.319969    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"
Sep 11 14:32:51.328152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.328108    2564 scope.go:117] "RemoveContainer" containerID="5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1"
Sep 11 14:32:51.328472 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:32:51.328443    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1\": container with ID starting with 5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1 not found: ID does not exist" containerID="5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1"
Sep 11 14:32:51.328579 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.328482    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1"} err="failed to get container status \"5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1\": rpc error: code = NotFound desc = could not find container \"5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1\": container with ID starting with 5aa40a75974d57109a8c87666f066e0b3d31813bdcdadfb51d0892149483a0f1 not found: ID does not exist"
Sep 11 14:32:51.328579 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.328505    2564 scope.go:117] "RemoveContainer" containerID="a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8"
Sep 11 14:32:51.336373 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.336358    2564 scope.go:117] "RemoveContainer" containerID="a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8"
Sep 11 14:32:51.336678 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:32:51.336604    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8\": container with ID starting with a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8 not found: ID does not exist" containerID="a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8"
Sep 11 14:32:51.336678 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.336647    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8"} err="failed to get container status \"a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8\": rpc error: code = NotFound desc = could not find container \"a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8\": container with ID starting with a114f405dd30d7f531f8f939176b135e3a98e5f2d16ab804ddfc387b1cc2e1f8 not found: ID does not exist"
Sep 11 14:32:51.347669 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.347645    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"]
Sep 11 14:32:51.351721 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.351696    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-031a56-749f8bc5c4-mspm9"]
Sep 11 14:32:51.369055 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.369034    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"]
Sep 11 14:32:51.372271 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:51.372249    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-031a56-654f46c7f9-s2zr2"]
Sep 11 14:32:52.275978 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:52.275938    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="130aaa7d-5fc3-40db-bbea-21b0a4beda7c" path="/var/lib/kubelet/pods/130aaa7d-5fc3-40db-bbea-21b0a4beda7c/volumes"
Sep 11 14:32:52.276545 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:32:52.276524    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81e8bf04-451a-40c0-a3e3-0827a1365196" path="/var/lib/kubelet/pods/81e8bf04-451a-40c0-a3e3-0827a1365196/volumes"
Sep 11 14:33:00.676637 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:00.676588    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-rxzgw"]
Sep 11 14:33:00.677054 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:00.676829    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw" podUID="fb99cd57-e8b3-493b-af8c-a61e6b450175" containerName="manager" containerID="cri-o://489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959" gracePeriod=30
Sep 11 14:33:00.928475 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:00.928421    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:33:01.081450 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.081417    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx9nx\" (UniqueName: \"kubernetes.io/projected/fb99cd57-e8b3-493b-af8c-a61e6b450175-kube-api-access-gx9nx\") pod \"fb99cd57-e8b3-493b-af8c-a61e6b450175\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") "
Sep 11 14:33:01.081615 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.081477    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-metrics-certs\") pod \"fb99cd57-e8b3-493b-af8c-a61e6b450175\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") "
Sep 11 14:33:01.081615 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.081510    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-webhook-cert\") pod \"fb99cd57-e8b3-493b-af8c-a61e6b450175\" (UID: \"fb99cd57-e8b3-493b-af8c-a61e6b450175\") "
Sep 11 14:33:01.083659 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.083612    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb99cd57-e8b3-493b-af8c-a61e6b450175-kube-api-access-gx9nx" (OuterVolumeSpecName: "kube-api-access-gx9nx") pod "fb99cd57-e8b3-493b-af8c-a61e6b450175" (UID: "fb99cd57-e8b3-493b-af8c-a61e6b450175"). InnerVolumeSpecName "kube-api-access-gx9nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:33:01.083659 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.083655    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fb99cd57-e8b3-493b-af8c-a61e6b450175" (UID: "fb99cd57-e8b3-493b-af8c-a61e6b450175"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:33:01.083833 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.083621    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fb99cd57-e8b3-493b-af8c-a61e6b450175" (UID: "fb99cd57-e8b3-493b-af8c-a61e6b450175"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:33:01.182866 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.182802    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx9nx\" (UniqueName: \"kubernetes.io/projected/fb99cd57-e8b3-493b-af8c-a61e6b450175-kube-api-access-gx9nx\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:33:01.182866 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.182828    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:33:01.182866 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.182836    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb99cd57-e8b3-493b-af8c-a61e6b450175-webhook-cert\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:33:01.371749 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.371711    2564 generic.go:356] "Generic (PLEG): container finished" podID="fb99cd57-e8b3-493b-af8c-a61e6b450175" containerID="489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959" exitCode=0
Sep 11 14:33:01.371905 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.371759    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw"
Sep 11 14:33:01.371905 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.371794    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw" event={"ID":"fb99cd57-e8b3-493b-af8c-a61e6b450175","Type":"ContainerDied","Data":"489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959"}
Sep 11 14:33:01.371905 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.371844    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-rxzgw" event={"ID":"fb99cd57-e8b3-493b-af8c-a61e6b450175","Type":"ContainerDied","Data":"ba3c09e41c9fe6e25a67c59b6776eb898d7553740e5b5a7e6e0947f733daa870"}
Sep 11 14:33:01.371905 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.371866    2564 scope.go:117] "RemoveContainer" containerID="489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959"
Sep 11 14:33:01.381699 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.381681    2564 scope.go:117] "RemoveContainer" containerID="489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959"
Sep 11 14:33:01.381980 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:33:01.381961    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959\": container with ID starting with 489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959 not found: ID does not exist" containerID="489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959"
Sep 11 14:33:01.382030 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.381989    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959"} err="failed to get container status \"489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959\": rpc error: code = NotFound desc = could not find container \"489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959\": container with ID starting with 489735d8caeeae01b6caeaa07ac5fe1eb453cc71db744aceb53d78b55be98959 not found: ID does not exist"
Sep 11 14:33:01.396283 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.396257    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-rxzgw"]
Sep 11 14:33:01.400110 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:01.400087    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-rxzgw"]
Sep 11 14:33:02.275148 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:33:02.275114    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb99cd57-e8b3-493b-af8c-a61e6b450175" path="/var/lib/kubelet/pods/fb99cd57-e8b3-493b-af8c-a61e6b450175/volumes"
Sep 11 14:34:02.285020 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.284987    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-4hz29"]
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285405    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd68485f-35c7-4ab1-9b16-25bb88d7afe4" containerName="payload-pre-processing"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285419    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd68485f-35c7-4ab1-9b16-25bb88d7afe4" containerName="payload-pre-processing"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285428    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78fc380c-d7af-44c1-92aa-9eac6f70110f" containerName="payload-processing"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285435    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="78fc380c-d7af-44c1-92aa-9eac6f70110f" containerName="payload-processing"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285447    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fef2b257-4e36-4d7d-a5a9-b6609f3a6957" containerName="maas-api"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285455    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fef2b257-4e36-4d7d-a5a9-b6609f3a6957" containerName="maas-api"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285468    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb99cd57-e8b3-493b-af8c-a61e6b450175" containerName="manager"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285473    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb99cd57-e8b3-493b-af8c-a61e6b450175" containerName="manager"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285479    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b858665b-a2b1-44af-af7f-70d0f4885f02" containerName="maas-api"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285484    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b858665b-a2b1-44af-af7f-70d0f4885f02" containerName="maas-api"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285490    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" containerName="maas-api"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285494    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" containerName="maas-api"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285504    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="130aaa7d-5fc3-40db-bbea-21b0a4beda7c" containerName="payload-pre-processing"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285511    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="130aaa7d-5fc3-40db-bbea-21b0a4beda7c" containerName="payload-pre-processing"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285521    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6a0ead7-3c0b-4928-beb5-199d8f698f77" containerName="payload-processing"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285529    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6a0ead7-3c0b-4928-beb5-199d8f698f77" containerName="payload-processing"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285537    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a82b62d5-9424-415b-97e3-1c984a936a6f" containerName="payload-pre-processing"
Sep 11 14:34:02.285534 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285543    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a82b62d5-9424-415b-97e3-1c984a936a6f" containerName="payload-pre-processing"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285554    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81e8bf04-451a-40c0-a3e3-0827a1365196" containerName="maas-api"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285559    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="81e8bf04-451a-40c0-a3e3-0827a1365196" containerName="maas-api"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285616    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="852fc4a4-7bad-4c3e-bc1a-6b2f89e19525" containerName="maas-api"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285636    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fef2b257-4e36-4d7d-a5a9-b6609f3a6957" containerName="maas-api"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285644    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="81e8bf04-451a-40c0-a3e3-0827a1365196" containerName="maas-api"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285650    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd68485f-35c7-4ab1-9b16-25bb88d7afe4" containerName="payload-pre-processing"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285655    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6a0ead7-3c0b-4928-beb5-199d8f698f77" containerName="payload-processing"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285662    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a82b62d5-9424-415b-97e3-1c984a936a6f" containerName="payload-pre-processing"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285669    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="78fc380c-d7af-44c1-92aa-9eac6f70110f" containerName="payload-processing"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285676    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="130aaa7d-5fc3-40db-bbea-21b0a4beda7c" containerName="payload-pre-processing"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285688    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb99cd57-e8b3-493b-af8c-a61e6b450175" containerName="manager"
Sep 11 14:34:02.286152 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.285695    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b858665b-a2b1-44af-af7f-70d0f4885f02" containerName="maas-api"
Sep 11 14:34:02.288842 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.288823    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.294711 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.294683    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 14:34:02.294831 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.294712    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 14:34:02.294831 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.294690    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 14:34:02.294831 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.294683    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dkzmk\""
Sep 11 14:34:02.313718 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.313693    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-4hz29"]
Sep 11 14:34:02.393304 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.393276    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjlbw\" (UniqueName: \"kubernetes.io/projected/b0315e1c-29a4-4045-b874-aa00d162711a-kube-api-access-tjlbw\") pod \"maas-controller-58c98bd7bd-4hz29\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") " pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.393458 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.393338    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-metrics-certs\") pod \"maas-controller-58c98bd7bd-4hz29\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") " pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.393458 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.393432    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-webhook-cert\") pod \"maas-controller-58c98bd7bd-4hz29\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") " pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.494572 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.494538    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-webhook-cert\") pod \"maas-controller-58c98bd7bd-4hz29\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") " pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.494747 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.494592    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjlbw\" (UniqueName: \"kubernetes.io/projected/b0315e1c-29a4-4045-b874-aa00d162711a-kube-api-access-tjlbw\") pod \"maas-controller-58c98bd7bd-4hz29\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") " pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.494747 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.494663    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-metrics-certs\") pod \"maas-controller-58c98bd7bd-4hz29\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") " pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.497068 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.497047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-webhook-cert\") pod \"maas-controller-58c98bd7bd-4hz29\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") " pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.497127 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.497104    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-metrics-certs\") pod \"maas-controller-58c98bd7bd-4hz29\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") " pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.506652 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.506605    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjlbw\" (UniqueName: \"kubernetes.io/projected/b0315e1c-29a4-4045-b874-aa00d162711a-kube-api-access-tjlbw\") pod \"maas-controller-58c98bd7bd-4hz29\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") " pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.598958 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.598928    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:02.891676 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.891653    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-4hz29"]
Sep 11 14:34:02.894159 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:34:02.894135    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0315e1c_29a4_4045_b874_aa00d162711a.slice/crio-344a4ba748266794d25261b357737c678e223f479846fa5af3eb67a53469cd3d WatchSource:0}: Error finding container 344a4ba748266794d25261b357737c678e223f479846fa5af3eb67a53469cd3d: Status 404 returned error can't find the container with id 344a4ba748266794d25261b357737c678e223f479846fa5af3eb67a53469cd3d
Sep 11 14:34:02.895405 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:02.895388    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 14:34:03.659067 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:03.658981    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-4hz29" event={"ID":"b0315e1c-29a4-4045-b874-aa00d162711a","Type":"ContainerStarted","Data":"7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd"}
Sep 11 14:34:03.659067 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:03.659033    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-4hz29" event={"ID":"b0315e1c-29a4-4045-b874-aa00d162711a","Type":"ContainerStarted","Data":"344a4ba748266794d25261b357737c678e223f479846fa5af3eb67a53469cd3d"}
Sep 11 14:34:03.659443 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:03.659096    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:03.696778 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:03.696733    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58c98bd7bd-4hz29" podStartSLOduration=1.187296466 podStartE2EDuration="1.696719861s" totalImagesPullingTime="509.423395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:34:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:34:03.695547001 +0000 UTC m=+2194.033122560" watchObservedRunningTime="2026-09-11 14:34:03.696719861 +0000 UTC m=+2194.034295420"
Sep 11 14:34:14.666535 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:14.666505    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:24.092287 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.092252    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-4hz29"]
Sep 11 14:34:24.092739 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.092439    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-58c98bd7bd-4hz29" podUID="b0315e1c-29a4-4045-b874-aa00d162711a" containerName="manager" containerID="cri-o://7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd" gracePeriod=30
Sep 11 14:34:24.334973 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.334949    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:24.391591 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.391514    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-metrics-certs\") pod \"b0315e1c-29a4-4045-b874-aa00d162711a\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") "
Sep 11 14:34:24.391591 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.391557    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjlbw\" (UniqueName: \"kubernetes.io/projected/b0315e1c-29a4-4045-b874-aa00d162711a-kube-api-access-tjlbw\") pod \"b0315e1c-29a4-4045-b874-aa00d162711a\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") "
Sep 11 14:34:24.391788 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.391662    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-webhook-cert\") pod \"b0315e1c-29a4-4045-b874-aa00d162711a\" (UID: \"b0315e1c-29a4-4045-b874-aa00d162711a\") "
Sep 11 14:34:24.393769 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.393739    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b0315e1c-29a4-4045-b874-aa00d162711a" (UID: "b0315e1c-29a4-4045-b874-aa00d162711a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:34:24.393910 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.393831    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b0315e1c-29a4-4045-b874-aa00d162711a" (UID: "b0315e1c-29a4-4045-b874-aa00d162711a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:34:24.394154 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.394135    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0315e1c-29a4-4045-b874-aa00d162711a-kube-api-access-tjlbw" (OuterVolumeSpecName: "kube-api-access-tjlbw") pod "b0315e1c-29a4-4045-b874-aa00d162711a" (UID: "b0315e1c-29a4-4045-b874-aa00d162711a"). InnerVolumeSpecName "kube-api-access-tjlbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:34:24.492922 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.492895    2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-webhook-cert\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:34:24.492922 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.492916    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0315e1c-29a4-4045-b874-aa00d162711a-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:34:24.492922 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.492926    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjlbw\" (UniqueName: \"kubernetes.io/projected/b0315e1c-29a4-4045-b874-aa00d162711a-kube-api-access-tjlbw\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:34:24.759212 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.759135    2564 generic.go:356] "Generic (PLEG): container finished" podID="b0315e1c-29a4-4045-b874-aa00d162711a" containerID="7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd" exitCode=0
Sep 11 14:34:24.759212 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.759180    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-4hz29"
Sep 11 14:34:24.759381 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.759217    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-4hz29" event={"ID":"b0315e1c-29a4-4045-b874-aa00d162711a","Type":"ContainerDied","Data":"7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd"}
Sep 11 14:34:24.759381 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.759271    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-4hz29" event={"ID":"b0315e1c-29a4-4045-b874-aa00d162711a","Type":"ContainerDied","Data":"344a4ba748266794d25261b357737c678e223f479846fa5af3eb67a53469cd3d"}
Sep 11 14:34:24.759381 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.759292    2564 scope.go:117] "RemoveContainer" containerID="7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd"
Sep 11 14:34:24.768596 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.768580    2564 scope.go:117] "RemoveContainer" containerID="7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd"
Sep 11 14:34:24.768878 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:34:24.768861    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd\": container with ID starting with 7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd not found: ID does not exist" containerID="7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd"
Sep 11 14:34:24.768993 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.768891    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd"} err="failed to get container status \"7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd\": rpc error: code = NotFound desc = could not find container \"7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd\": container with ID starting with 7a2f1bee045d79d8f9136b2d3f48217135b18eaf577f2bfe38f8b94bfff322dd not found: ID does not exist"
Sep 11 14:34:24.782244 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.782219    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-4hz29"]
Sep 11 14:34:24.787502 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:24.787476    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-4hz29"]
Sep 11 14:34:25.635311 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.635276    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-94b97"]
Sep 11 14:34:25.635700 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.635638    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0315e1c-29a4-4045-b874-aa00d162711a" containerName="manager"
Sep 11 14:34:25.635700 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.635650    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0315e1c-29a4-4045-b874-aa00d162711a" containerName="manager"
Sep 11 14:34:25.635700 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.635702    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0315e1c-29a4-4045-b874-aa00d162711a" containerName="manager"
Sep 11 14:34:25.639853 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.639837    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.645007 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.644964    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 14:34:25.645007 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.644995    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 14:34:25.645177 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.645011    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dkzmk\""
Sep 11 14:34:25.645177 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.645015    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 14:34:25.656040 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.656015    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-94b97"]
Sep 11 14:34:25.701827 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.701793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqs67\" (UniqueName: \"kubernetes.io/projected/486370e2-9246-435e-8712-147d6ce6b369-kube-api-access-wqs67\") pod \"maas-controller-58c98bd7bd-94b97\" (UID: \"486370e2-9246-435e-8712-147d6ce6b369\") " pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.701995 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.701834    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486370e2-9246-435e-8712-147d6ce6b369-metrics-certs\") pod \"maas-controller-58c98bd7bd-94b97\" (UID: \"486370e2-9246-435e-8712-147d6ce6b369\") " pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.701995 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.701860    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486370e2-9246-435e-8712-147d6ce6b369-webhook-cert\") pod \"maas-controller-58c98bd7bd-94b97\" (UID: \"486370e2-9246-435e-8712-147d6ce6b369\") " pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.802905 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.802878    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqs67\" (UniqueName: \"kubernetes.io/projected/486370e2-9246-435e-8712-147d6ce6b369-kube-api-access-wqs67\") pod \"maas-controller-58c98bd7bd-94b97\" (UID: \"486370e2-9246-435e-8712-147d6ce6b369\") " pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.803070 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.802914    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486370e2-9246-435e-8712-147d6ce6b369-metrics-certs\") pod \"maas-controller-58c98bd7bd-94b97\" (UID: \"486370e2-9246-435e-8712-147d6ce6b369\") " pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.803070 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.802933    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486370e2-9246-435e-8712-147d6ce6b369-webhook-cert\") pod \"maas-controller-58c98bd7bd-94b97\" (UID: \"486370e2-9246-435e-8712-147d6ce6b369\") " pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.805337 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.805306    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486370e2-9246-435e-8712-147d6ce6b369-metrics-certs\") pod \"maas-controller-58c98bd7bd-94b97\" (UID: \"486370e2-9246-435e-8712-147d6ce6b369\") " pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.805443 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.805370    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/486370e2-9246-435e-8712-147d6ce6b369-webhook-cert\") pod \"maas-controller-58c98bd7bd-94b97\" (UID: \"486370e2-9246-435e-8712-147d6ce6b369\") " pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.817200 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.817177    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqs67\" (UniqueName: \"kubernetes.io/projected/486370e2-9246-435e-8712-147d6ce6b369-kube-api-access-wqs67\") pod \"maas-controller-58c98bd7bd-94b97\" (UID: \"486370e2-9246-435e-8712-147d6ce6b369\") " pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:25.949868 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:25.949797    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:26.228198 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:26.228166    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-58c98bd7bd-94b97"]
Sep 11 14:34:26.230256 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:34:26.230234    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod486370e2_9246_435e_8712_147d6ce6b369.slice/crio-08c037098c8cbf2c5a0c603ddde64b6d4438deea2a88f17392411f53b5cb555a WatchSource:0}: Error finding container 08c037098c8cbf2c5a0c603ddde64b6d4438deea2a88f17392411f53b5cb555a: Status 404 returned error can't find the container with id 08c037098c8cbf2c5a0c603ddde64b6d4438deea2a88f17392411f53b5cb555a
Sep 11 14:34:26.275292 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:26.275263    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0315e1c-29a4-4045-b874-aa00d162711a" path="/var/lib/kubelet/pods/b0315e1c-29a4-4045-b874-aa00d162711a/volumes"
Sep 11 14:34:26.771921 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:26.771888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-94b97" event={"ID":"486370e2-9246-435e-8712-147d6ce6b369","Type":"ContainerStarted","Data":"6ac5eb6d136eb00dfb4ad0644b4edd689a281ec2003f7f362e430832bdc72df0"}
Sep 11 14:34:26.772369 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:26.771939    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-58c98bd7bd-94b97" event={"ID":"486370e2-9246-435e-8712-147d6ce6b369","Type":"ContainerStarted","Data":"08c037098c8cbf2c5a0c603ddde64b6d4438deea2a88f17392411f53b5cb555a"}
Sep 11 14:34:26.772369 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:26.772039    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:34:26.800308 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:26.800262    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-58c98bd7bd-94b97" podStartSLOduration=1.432719619 podStartE2EDuration="1.800244988s" totalImagesPullingTime="367.525369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:34:26.799076797 +0000 UTC m=+2217.136652358" watchObservedRunningTime="2026-09-11 14:34:26.800244988 +0000 UTC m=+2217.137820547"
Sep 11 14:34:37.779692 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:34:37.779661    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-58c98bd7bd-94b97"
Sep 11 14:37:56.150969 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.150892    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"]
Sep 11 14:37:56.154389 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.154368    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.172128 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.172105    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"]
Sep 11 14:37:56.186036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.186002    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdef5a4f-1988-4129-9669-aa2c8816612a-openshift-service-ca\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.186036 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.186037    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdef5a4f-1988-4129-9669-aa2c8816612a-metrics-certs\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.186233 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.186211    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdef5a4f-1988-4129-9669-aa2c8816612a-maas-api-tls\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.186286 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.186244    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcdxq\" (UniqueName: \"kubernetes.io/projected/fdef5a4f-1988-4129-9669-aa2c8816612a-kube-api-access-wcdxq\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.287172 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.287141    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdef5a4f-1988-4129-9669-aa2c8816612a-maas-api-tls\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.287172 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.287176    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcdxq\" (UniqueName: \"kubernetes.io/projected/fdef5a4f-1988-4129-9669-aa2c8816612a-kube-api-access-wcdxq\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.287382 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.287198    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdef5a4f-1988-4129-9669-aa2c8816612a-openshift-service-ca\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.287382 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.287219    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdef5a4f-1988-4129-9669-aa2c8816612a-metrics-certs\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.287805 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.287782    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdef5a4f-1988-4129-9669-aa2c8816612a-openshift-service-ca\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.289604 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.289581    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdef5a4f-1988-4129-9669-aa2c8816612a-maas-api-tls\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.289741 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.289722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdef5a4f-1988-4129-9669-aa2c8816612a-metrics-certs\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.306309 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.306282    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcdxq\" (UniqueName: \"kubernetes.io/projected/fdef5a4f-1988-4129-9669-aa2c8816612a-kube-api-access-wcdxq\") pod \"maas-api-7959f48cf9-dgqr9\" (UID: \"fdef5a4f-1988-4129-9669-aa2c8816612a\") " pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.464621 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.464537    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:56.738892 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:56.738757    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"]
Sep 11 14:37:56.741541 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:37:56.741516    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdef5a4f_1988_4129_9669_aa2c8816612a.slice/crio-24d8f1d8ea1c0a692ad0ec49a4b154a99889981f08cca3bd38bbaa671ea38479 WatchSource:0}: Error finding container 24d8f1d8ea1c0a692ad0ec49a4b154a99889981f08cca3bd38bbaa671ea38479: Status 404 returned error can't find the container with id 24d8f1d8ea1c0a692ad0ec49a4b154a99889981f08cca3bd38bbaa671ea38479
Sep 11 14:37:57.739286 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:57.739198    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9" event={"ID":"fdef5a4f-1988-4129-9669-aa2c8816612a","Type":"ContainerStarted","Data":"7749c26346243be37b7d60707cc39586682b360a506eb0b817b5d91e60cab660"}
Sep 11 14:37:57.739286 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:57.739248    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9" event={"ID":"fdef5a4f-1988-4129-9669-aa2c8816612a","Type":"ContainerStarted","Data":"24d8f1d8ea1c0a692ad0ec49a4b154a99889981f08cca3bd38bbaa671ea38479"}
Sep 11 14:37:57.739708 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:57.739286    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:37:57.761498 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:37:57.761450    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9" podStartSLOduration=1.196740206 podStartE2EDuration="1.761435839s" totalImagesPullingTime="564.695633ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 14:37:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:37:57.759529349 +0000 UTC m=+2428.097104909" watchObservedRunningTime="2026-09-11 14:37:57.761435839 +0000 UTC m=+2428.099011399"
Sep 11 14:38:03.748777 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:03.748744    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7959f48cf9-dgqr9"
Sep 11 14:38:03.804621 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:03.804592    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"]
Sep 11 14:38:03.804876 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:03.804852    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh" podUID="f89b66b3-7024-4432-884a-e5ddd4cf26bb" containerName="maas-api" containerID="cri-o://f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718" gracePeriod=30
Sep 11 14:38:04.354693 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:04.354657    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh" podUID="f89b66b3-7024-4432-884a-e5ddd4cf26bb" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.31:8443/health\": dial tcp 10.132.0.31:8443: connect: connection refused"
Sep 11 14:38:05.049869 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.049845    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:38:05.168956 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.168887    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f89b66b3-7024-4432-884a-e5ddd4cf26bb-openshift-service-ca\") pod \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") "
Sep 11 14:38:05.168956 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.168938    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89xmr\" (UniqueName: \"kubernetes.io/projected/f89b66b3-7024-4432-884a-e5ddd4cf26bb-kube-api-access-89xmr\") pod \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") "
Sep 11 14:38:05.169182 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.168973    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-maas-api-tls\") pod \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") "
Sep 11 14:38:05.169182 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.169035    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-metrics-certs\") pod \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\" (UID: \"f89b66b3-7024-4432-884a-e5ddd4cf26bb\") "
Sep 11 14:38:05.169299 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.169223    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f89b66b3-7024-4432-884a-e5ddd4cf26bb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f89b66b3-7024-4432-884a-e5ddd4cf26bb" (UID: "f89b66b3-7024-4432-884a-e5ddd4cf26bb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 14:38:05.171174 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.171142    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f89b66b3-7024-4432-884a-e5ddd4cf26bb" (UID: "f89b66b3-7024-4432-884a-e5ddd4cf26bb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:38:05.171174 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.171151    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f89b66b3-7024-4432-884a-e5ddd4cf26bb-kube-api-access-89xmr" (OuterVolumeSpecName: "kube-api-access-89xmr") pod "f89b66b3-7024-4432-884a-e5ddd4cf26bb" (UID: "f89b66b3-7024-4432-884a-e5ddd4cf26bb"). InnerVolumeSpecName "kube-api-access-89xmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 14:38:05.171325 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.171219    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f89b66b3-7024-4432-884a-e5ddd4cf26bb" (UID: "f89b66b3-7024-4432-884a-e5ddd4cf26bb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 14:38:05.269853 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.269822    2564 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-metrics-certs\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:38:05.269853 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.269848    2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f89b66b3-7024-4432-884a-e5ddd4cf26bb-openshift-service-ca\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:38:05.270032 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.269862    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89xmr\" (UniqueName: \"kubernetes.io/projected/f89b66b3-7024-4432-884a-e5ddd4cf26bb-kube-api-access-89xmr\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:38:05.270032 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.269875    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f89b66b3-7024-4432-884a-e5ddd4cf26bb-maas-api-tls\") on node \"ip-10-0-129-195.ec2.internal\" DevicePath \"\""
Sep 11 14:38:05.777686 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.777650    2564 generic.go:356] "Generic (PLEG): container finished" podID="f89b66b3-7024-4432-884a-e5ddd4cf26bb" containerID="f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718" exitCode=0
Sep 11 14:38:05.777862 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.777702    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"
Sep 11 14:38:05.777862 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.777717    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh" event={"ID":"f89b66b3-7024-4432-884a-e5ddd4cf26bb","Type":"ContainerDied","Data":"f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718"}
Sep 11 14:38:05.777862 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.777795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh" event={"ID":"f89b66b3-7024-4432-884a-e5ddd4cf26bb","Type":"ContainerDied","Data":"ab16f8b6d4f0ccdf5e2c68e937790f35754883ceffdc48c9aeb533a93c16f4ad"}
Sep 11 14:38:05.777862 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.777818    2564 scope.go:117] "RemoveContainer" containerID="f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718"
Sep 11 14:38:05.786660 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.786619    2564 scope.go:117] "RemoveContainer" containerID="f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718"
Sep 11 14:38:05.786938 ip-10-0-129-195 kubenswrapper[2564]: E0911 14:38:05.786918    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718\": container with ID starting with f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718 not found: ID does not exist" containerID="f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718"
Sep 11 14:38:05.786997 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.786945    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718"} err="failed to get container status \"f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718\": rpc error: code = NotFound desc = could not find container \"f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718\": container with ID starting with f5059f11af2c6bc955901c18813233af6b96f61fb4b9088331f7873ea0bf4718 not found: ID does not exist"
Sep 11 14:38:05.801889 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.801867    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"]
Sep 11 14:38:05.807939 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:05.807917    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-d8d795dc5-z6bxh"]
Sep 11 14:38:06.275700 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:06.275667    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f89b66b3-7024-4432-884a-e5ddd4cf26bb" path="/var/lib/kubelet/pods/f89b66b3-7024-4432-884a-e5ddd4cf26bb/volumes"
Sep 11 14:38:31.922326 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:31.922292    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58c98bd7bd-94b97_486370e2-9246-435e-8712-147d6ce6b369/manager/0.log"
Sep 11 14:38:32.050533 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:32.050508    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cx8rc_5d07caad-64bc-46d3-9b3d-96a7c3c69e24/server/0.log"
Sep 11 14:38:32.168535 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:32.168507    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kzl4k_c905ec0d-2626-4e38-918d-df403ab57f76/manager/0.log"
Sep 11 14:38:32.292956 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:32.292886    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5586bcdcf6-f77wt_2db6f35e-1e39-4396-a604-ab0569226528/copy-manifests/0.log"
Sep 11 14:38:32.299835 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:32.299811    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5586bcdcf6-f77wt_2db6f35e-1e39-4396-a604-ab0569226528/manager/0.log"
Sep 11 14:38:32.920726 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:32.920692    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7959f48cf9-dgqr9_fdef5a4f-1988-4129-9669-aa2c8816612a/maas-api/0.log"
Sep 11 14:38:35.501341 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:35.501312    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7x9t7_d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7/discovery/0.log"
Sep 11 14:38:36.007279 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:36.007249    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-4jzpw_70ee0d68-d5ee-4abe-8b6b-80142836b2aa/payload-pre-processing/0.log"
Sep 11 14:38:36.608879 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:36.608835    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4kvwz_be65d086-8894-407a-bad9-6421ec87abeb/main/0.log"
Sep 11 14:38:37.106955 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:37.106922    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvq6s5_b1913686-57ef-458e-b84a-990becea2844/main/0.log"
Sep 11 14:38:45.667560 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:45.667522    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vz6bg_9953ef84-7f0d-4d91-8f62-735ee11380b3/global-pull-secret-syncer/0.log"
Sep 11 14:38:45.774747 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:45.774714    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ljzn7_4ff248cb-b2cd-46e6-865f-c5d26df89969/konnectivity-agent/0.log"
Sep 11 14:38:45.844539 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:45.844506    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-195.ec2.internal_1a193ec36d49eb843a04da4a94012319/haproxy/0.log"
Sep 11 14:38:52.515645 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.515596    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_46703f2e-56d2-4601-b9b3-f1b634d6a0ed/alertmanager/0.log"
Sep 11 14:38:52.549124 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.549098    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_46703f2e-56d2-4601-b9b3-f1b634d6a0ed/config-reloader/0.log"
Sep 11 14:38:52.574035 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.574013    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_46703f2e-56d2-4601-b9b3-f1b634d6a0ed/kube-rbac-proxy-web/0.log"
Sep 11 14:38:52.602470 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.602447    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_46703f2e-56d2-4601-b9b3-f1b634d6a0ed/kube-rbac-proxy/0.log"
Sep 11 14:38:52.627872 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.627846    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_46703f2e-56d2-4601-b9b3-f1b634d6a0ed/kube-rbac-proxy-metric/0.log"
Sep 11 14:38:52.653403 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.653375    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_46703f2e-56d2-4601-b9b3-f1b634d6a0ed/prom-label-proxy/0.log"
Sep 11 14:38:52.677728 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.677702    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_46703f2e-56d2-4601-b9b3-f1b634d6a0ed/init-config-reloader/0.log"
Sep 11 14:38:52.725127 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.725093    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-vjvrz_b357ccb1-6082-4553-8552-322e646338da/cluster-monitoring-operator/0.log"
Sep 11 14:38:52.757787 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.757756    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-hbcz5_1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c/kube-state-metrics/0.log"
Sep 11 14:38:52.781599 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.781538    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-hbcz5_1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c/kube-rbac-proxy-main/0.log"
Sep 11 14:38:52.809096 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:52.809076    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-hbcz5_1aef610a-7ea0-477f-97a5-7f8ea0c4fc6c/kube-rbac-proxy-self/0.log"
Sep 11 14:38:53.262765 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:53.262737    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l8fs8_ce92e2e1-1ebc-4f45-afda-0d7f2086b883/node-exporter/0.log"
Sep 11 14:38:53.293898 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:53.293866    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l8fs8_ce92e2e1-1ebc-4f45-afda-0d7f2086b883/kube-rbac-proxy/0.log"
Sep 11 14:38:53.325207 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:53.325150    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l8fs8_ce92e2e1-1ebc-4f45-afda-0d7f2086b883/init-textfile/0.log"
Sep 11 14:38:53.788257 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:53.788183    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-584fd8cb7d-h4jqw_b2fd31ce-a1f8-4be4-98ba-e445464cb531/telemeter-client/0.log"
Sep 11 14:38:53.809422 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:53.809402    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-584fd8cb7d-h4jqw_b2fd31ce-a1f8-4be4-98ba-e445464cb531/reload/0.log"
Sep 11 14:38:53.836064 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:53.836041    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-584fd8cb7d-h4jqw_b2fd31ce-a1f8-4be4-98ba-e445464cb531/kube-rbac-proxy/0.log"
Sep 11 14:38:55.295593 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.295567    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-r4547_c4e76c34-0414-47ad-971f-2e3aaad3ec4a/networking-console-plugin/0.log"
Sep 11 14:38:55.624977 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.624949    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"]
Sep 11 14:38:55.625331 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.625309    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f89b66b3-7024-4432-884a-e5ddd4cf26bb" containerName="maas-api"
Sep 11 14:38:55.625331 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.625330    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f89b66b3-7024-4432-884a-e5ddd4cf26bb" containerName="maas-api"
Sep 11 14:38:55.625547 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.625495    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f89b66b3-7024-4432-884a-e5ddd4cf26bb" containerName="maas-api"
Sep 11 14:38:55.628579 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.628562    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.631622 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.631603    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ktch4\"/\"kube-root-ca.crt\""
Sep 11 14:38:55.631747 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.631709    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ktch4\"/\"openshift-service-ca.crt\""
Sep 11 14:38:55.632419 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.632402    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ktch4\"/\"default-dockercfg-mxzzh\""
Sep 11 14:38:55.639252 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.639230    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"]
Sep 11 14:38:55.707246 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.707214    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-podres\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.707388 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.707311    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-proc\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.707388 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.707343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-lib-modules\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.707491 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.707374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79ckp\" (UniqueName: \"kubernetes.io/projected/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-kube-api-access-79ckp\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.707491 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.707424    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-sys\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.808318 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.808284    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-podres\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.808502 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.808335    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-proc\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.808502 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.808351    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-lib-modules\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.808502 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.808430    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-proc\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.808502 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.808448    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-podres\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.808502 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.808483    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79ckp\" (UniqueName: \"kubernetes.io/projected/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-kube-api-access-79ckp\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.808774 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.808508    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-lib-modules\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.808774 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.808530    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-sys\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.808774 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.808657    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-sys\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.822316 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.822295    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79ckp\" (UniqueName: \"kubernetes.io/projected/d61d1261-11ed-42d5-9e0b-84ea1baf76e3-kube-api-access-79ckp\") pod \"perf-node-gather-daemonset-xtkt9\" (UID: \"d61d1261-11ed-42d5-9e0b-84ea1baf76e3\") " pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:55.938105 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:55.938026    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:56.210941 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:56.210917    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"]
Sep 11 14:38:56.212452 ip-10-0-129-195 kubenswrapper[2564]: W0911 14:38:56.212422    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd61d1261_11ed_42d5_9e0b_84ea1baf76e3.slice/crio-6124ea23d5312d033fed0e804feadda51726eb5dfbd41b039699de864bf9969f WatchSource:0}: Error finding container 6124ea23d5312d033fed0e804feadda51726eb5dfbd41b039699de864bf9969f: Status 404 returned error can't find the container with id 6124ea23d5312d033fed0e804feadda51726eb5dfbd41b039699de864bf9969f
Sep 11 14:38:57.013653 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:57.013604    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9" event={"ID":"d61d1261-11ed-42d5-9e0b-84ea1baf76e3","Type":"ContainerStarted","Data":"b4b4c1c4c81ae7208cde516a5e5833f04ecb453dbaf80045c5965b43a6d4b877"}
Sep 11 14:38:57.014101 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:57.013678    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9" event={"ID":"d61d1261-11ed-42d5-9e0b-84ea1baf76e3","Type":"ContainerStarted","Data":"6124ea23d5312d033fed0e804feadda51726eb5dfbd41b039699de864bf9969f"}
Sep 11 14:38:57.014101 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:57.013708    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:38:57.033514 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:57.033467    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9" podStartSLOduration=2.033451636 podStartE2EDuration="2.033451636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 14:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 14:38:57.031822796 +0000 UTC m=+2487.369398355" watchObservedRunningTime="2026-09-11 14:38:57.033451636 +0000 UTC m=+2487.371027195"
Sep 11 14:38:58.546561 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:58.546536    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p7bff_d6a7405c-5ea0-49dc-a61b-c3397ec12438/dns/0.log"
Sep 11 14:38:58.571367 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:58.571345    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p7bff_d6a7405c-5ea0-49dc-a61b-c3397ec12438/kube-rbac-proxy/0.log"
Sep 11 14:38:58.651920 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:58.651896    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2zmjn_bc389e8a-e4aa-4f96-a52b-2bf944c5bfaa/dns-node-resolver/0.log"
Sep 11 14:38:59.430200 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:38:59.430170    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hjsjr_00dc4e9d-f4e7-41a6-9d88-3411a8df5400/node-ca/0.log"
Sep 11 14:39:00.823468 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:00.823438    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7x9t7_d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7/discovery/0.log"
Sep 11 14:39:00.962013 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:00.961986    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-4jzpw_70ee0d68-d5ee-4abe-8b6b-80142836b2aa/payload-pre-processing/0.log"
Sep 11 14:39:01.964458 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:01.964431    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k6t77_2af3aa4c-d665-483e-874c-0d085d6d1da9/serve-healthcheck-canary/0.log"
Sep 11 14:39:02.567153 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:02.567122    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-45kfw_1f4e7bd1-2741-48b2-a850-7ae70e22ad87/kube-rbac-proxy/0.log"
Sep 11 14:39:02.604398 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:02.604377    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-45kfw_1f4e7bd1-2741-48b2-a850-7ae70e22ad87/exporter/0.log"
Sep 11 14:39:02.629496 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:02.629471    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-45kfw_1f4e7bd1-2741-48b2-a850-7ae70e22ad87/extractor/0.log"
Sep 11 14:39:03.027892 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:03.027819    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ktch4/perf-node-gather-daemonset-xtkt9"
Sep 11 14:39:05.669830 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:05.669795    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-58c98bd7bd-94b97_486370e2-9246-435e-8712-147d6ce6b369/manager/0.log"
Sep 11 14:39:05.692288 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:05.692267    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-cx8rc_5d07caad-64bc-46d3-9b3d-96a7c3c69e24/server/0.log"
Sep 11 14:39:05.781417 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:05.781389    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kzl4k_c905ec0d-2626-4e38-918d-df403ab57f76/manager/0.log"
Sep 11 14:39:05.807668 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:05.807619    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5586bcdcf6-f77wt_2db6f35e-1e39-4396-a604-ab0569226528/manager/0.log"
Sep 11 14:39:05.834516 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:05.834496    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5586bcdcf6-f77wt_2db6f35e-1e39-4396-a604-ab0569226528/copy-manifests/0.log"
Sep 11 14:39:07.181913 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:07.181834    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7x9t7_d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7/discovery/0.log"
Sep 11 14:39:07.277484 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:07.277440    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-4jzpw_70ee0d68-d5ee-4abe-8b6b-80142836b2aa/payload-pre-processing/0.log"
Sep 11 14:39:07.421620 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:07.421595    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-779774756-xjsrj_6548a881-f3ea-43ff-bb08-36a48be1dff6/manager/0.log"
Sep 11 14:39:07.454771 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:07.454693    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-ttsph_5e0e64d3-36c3-4b71-b133-cf64bed28532/openshift-lws-operator/0.log"
Sep 11 14:39:09.155983 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:09.155921    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-7x9t7_d186a0c3-98ab-4c8a-be6d-7dc1d0bceee7/discovery/0.log"
Sep 11 14:39:09.267658 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:09.267601    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-4jzpw_70ee0d68-d5ee-4abe-8b6b-80142836b2aa/payload-pre-processing/0.log"
Sep 11 14:39:13.039584 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:13.039551    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-p58zp_4a8c46b7-5901-4f1e-b98b-9197917b7656/kube-storage-version-migrator-operator/1.log"
Sep 11 14:39:13.041974 ip-10-0-129-195 kubenswrapper[2564]: I0911 14:39:13.040493    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-p58zp_4a8c46b7-5901-4f1e-b98b-9197917b7656/kube-storage-version-migrator-operator/0.log"