Sep 16 18:14:11.839268 ip-10-0-129-142 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 18:14:11.839280 ip-10-0-129-142 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 18:14:11.839288 ip-10-0-129-142 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 18:14:11.839488 ip-10-0-129-142 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 18:14:22.064293 ip-10-0-129-142 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 18:14:22.064308 ip-10-0-129-142 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c0e1cf0210204d4abe805ba0cb8af1c0 --
Sep 16 18:16:16.315476 ip-10-0-129-142 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 18:16:16.747936 ip-10-0-129-142 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:16:16.747936 ip-10-0-129-142 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 18:16:16.747936 ip-10-0-129-142 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:16:16.747936 ip-10-0-129-142 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 18:16:16.747936 ip-10-0-129-142 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 18:16:16.749798 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.749719    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 18:16:16.752605 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752590    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:16.752605 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752605    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752609    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752611    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752614    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752617    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752619    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752622    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752625    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752627    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752630    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752633    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752645    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752648    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752650    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752653    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752655    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752658    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752661    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752663    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752667    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:16.752665 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752670    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752673    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752676    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752679    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752682    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752684    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752687    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752689    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752692    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752694    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752697    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752699    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752702    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752705    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752707    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752710    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752712    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752715    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752717    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:16.753140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752719    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752724    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752728    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752730    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752733    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752737    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752739    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752742    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752744    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752747    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752749    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752751    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752754    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752757    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752759    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752762    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752765    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752768    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752770    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:16.753730 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752773    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752775    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752778    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752780    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752784    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752787    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752790    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752793    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752796    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752798    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752801    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752803    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752806    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752808    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752810    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752813    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752815    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752818    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752820    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752823    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:16.754412 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752826    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752829    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752831    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752834    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752836    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752840    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752842    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752845    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752849    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752852    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752854    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752857    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752859    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752862    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752865    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752869    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752871    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752874    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752876    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:16.754940 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.752879    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:16.755822 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755789    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:16.755934 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755822    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:16.755934 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755836    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:16.755934 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755840    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:16.755934 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755846    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755958    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755966    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755970    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755972    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755975    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755978    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755981    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755984    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755987    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755989    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755992    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755995    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.755997    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756000    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756002    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756005    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756008    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756010    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756013    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:16.756147 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756015    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756018    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756020    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756023    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756025    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756028    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756030    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756033    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756035    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756037    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756040    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756043    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756048    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756060    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756064    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756067    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756069    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756072    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756075    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756078    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:16.756672 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756080    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756083    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756086    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756088    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756091    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756094    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756096    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756099    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756102    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756105    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756108    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756111    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756114    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756116    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756119    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756122    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756124    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756126    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756129    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756132    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:16.757193 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756134    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756136    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756139    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756141    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756144    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756147    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756149    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756152    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756155    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756157    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756160    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756163    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756165    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756168    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756171    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756175    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756177    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756180    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756183    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756185    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:16.757699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756188    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756191    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756194    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756197    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756199    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756202    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756221    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756224    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756227    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756231    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756236    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756239    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756244    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756248    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756251    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756634    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756639    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756643    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756647    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:16.758186 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756650    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756652    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756655    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756658    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756661    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756665    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756669    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756672    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756675    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756678    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756681    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756684    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756687    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756689    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756694    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756697    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756699    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756702    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756704    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756707    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:16.758682 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756710    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756712    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756715    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756717    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756720    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756723    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756725    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756727    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756730    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756732    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756735    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756738    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756741    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756744    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756746    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756749    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756751    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756754    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756756    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756759    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:16.759171 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756763    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756765    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756768    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756770    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756773    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756776    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756779    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756782    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756784    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756787    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756789    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756792    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756794    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756797    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756799    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756802    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756805    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756807    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756809    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756812    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:16.759700 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756814    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756817    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756819    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756822    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756825    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756827    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756832    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756835    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756839    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756842    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756845    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756848    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756851    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756854    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756856    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756859    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756861    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756863    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756867    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:16.760188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756870    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756872    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756875    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756877    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756880    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756882    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756885    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756887    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756890    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756892    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756895    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756897    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756899    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756902    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756905    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.756907    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.756973    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757011    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757019    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757023    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757027    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 18:16:16.760686 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757030    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757035    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757039    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757042    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757045    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757048    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757052    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757055    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757058    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757061    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757066    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757069    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757072    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757076    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757079    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757082    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757085    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757088    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757092    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757095    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757097    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757101    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757104    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757107    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 18:16:16.761244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757110    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757113    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757115    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757120    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757123    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757126    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757129    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757132    2576 flags.go:64] FLAG: --enable-server="true"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757135    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757139    2576 flags.go:64] FLAG: --event-burst="100"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757142    2576 flags.go:64] FLAG: --event-qps="50"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757145    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757148    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757150    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757155    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757157    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757160    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757163    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757168    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757171    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757174    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757177    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757180    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757183    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757186    2576 flags.go:64] FLAG: --feature-gates=""
Sep 16 18:16:16.761838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757190    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757193    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757196    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757199    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757202    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757220    2576 flags.go:64] FLAG: --help="false"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757225    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-129-142.ec2.internal"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757230    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757234    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757237    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757241    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757244    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757247    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757250    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757253    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757256    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757259    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757262    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757264    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757267    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757270    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757273    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757276    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757279    2576 flags.go:64] FLAG: --lock-file=""
Sep 16 18:16:16.762520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757282    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757285    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757288    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757294    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757298    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757301    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757303    2576 flags.go:64] FLAG: --logging-format="text"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757306    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757310    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757312    2576 flags.go:64] FLAG: --manifest-url=""
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757315    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757319    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757322    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757326    2576 flags.go:64] FLAG: --max-pods="110"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757329    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757332    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757335    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757338    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757341    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757344    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757347    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757355    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757358    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757361    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 18:16:16.763114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757364    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757367    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757371    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757374    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757377    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757380    2576 flags.go:64] FLAG: --port="10250"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757383    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757386    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bb5442b31b6677de"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757389    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757392    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757395    2576 flags.go:64] FLAG: --register-node="true"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757398    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757401    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757405    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757408    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757411    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757415    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757418    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757421    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757424    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757427    2576 flags.go:64] FLAG: --runonce="false"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757430    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757433    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757436    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757439    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 18:16:16.763759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757442    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757447    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757450    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757453    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757456    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757459    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757462    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757465    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757468    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757471    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757474    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757479    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757482    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757485    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757488    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757491    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757494    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757497    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757500    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757502    2576 flags.go:64] FLAG: --v="2"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757506    2576 flags.go:64] FLAG: --version="false"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757510    2576 flags.go:64] FLAG: --vmodule=""
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757514    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.757517    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757632    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:16.764393 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757635    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757638    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757641    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757643    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757646    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757649    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757652    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757654    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757658    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757661    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757663    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757666    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757668    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757671    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757674    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757677    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757679    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757682    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757684    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757687    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:16.765040 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757690    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757692    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757695    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757697    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757700    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757703    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757706    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757709    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757711    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757714    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757719    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757722    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757724    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757727    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757729    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757732    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757735    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757737    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757740    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757742    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:16.765560 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757747    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757749    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757754    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757758    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757761    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757764    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757767    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757770    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757773    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757776    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757779    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757781    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757785    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757787    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757790    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757793    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757795    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757798    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757801    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:16.766048 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757803    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757806    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757808    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757813    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757816    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757818    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757821    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757824    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757826    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757829    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757831    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757834    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757836    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757840    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757843    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757845    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757848    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757850    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757853    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757856    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:16.766521 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757858    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757861    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757863    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757866    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757868    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757872    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757876    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757879    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757882    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757885    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757888    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757890    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757894    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757897    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757900    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757903    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757906    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757909    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:16.767011 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.757912    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.758413    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.766046    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.766063    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.766900    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.766906    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766961    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766966    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766970    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766973    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766976    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766978    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766981    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766984    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766987    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:16.767477 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766989    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766992    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766995    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.766998    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767000    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767003    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767005    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767008    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767011    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767014    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767017    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767020    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767023    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767027    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767031    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767035    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767038    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767041    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767043    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:16.767857 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767046    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767050    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767053    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767056    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767058    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767061    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767064    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767067    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767069    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767072    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767074    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767077    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767079    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767082    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767084    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767087    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767090    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767093    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767095    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767098    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:16.768431 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767101    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767104    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767107    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767109    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767112    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767114    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767117    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767120    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767123    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767126    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767129    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767132    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767135    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767138    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767141    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767143    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767146    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767149    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767151    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767154    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:16.768909 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767156    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767159    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767162    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767164    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767167    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767169    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767172    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767174    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767177    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767179    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767182    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767184    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767187    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767189    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767193    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767196    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767199    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767201    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767204    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:16.769397 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767225    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767227    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767230    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767233    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767236    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767238    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767242    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767245    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767249    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767252    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767255    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767257    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.767262    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767373    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767378    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 18:16:16.769856 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767381    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767383    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767386    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767389    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767392    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767394    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767397    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767399    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767402    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767405    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767408    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767410    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767413    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767415    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767418    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767421    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767423    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767427    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767431    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767434    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 18:16:16.770244 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767437    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767439    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767442    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767445    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767448    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767457    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767462    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767464    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767467    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767469    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767472    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767474    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767478    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767480    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767483    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767486    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767489    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767491    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767494    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767496    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 18:16:16.770733 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767499    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767503    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767507    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767510    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767512    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767515    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767517    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767520    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767522    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767525    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767527    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767530    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767532    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767535    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767537    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767541    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767545    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767548    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767551    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 18:16:16.771227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767554    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767557    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767560    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767562    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767564    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767567    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767570    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767572    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767575    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767577    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767580    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767582    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767585    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767587    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767590    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767592    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767595    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767597    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767600    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767602    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 18:16:16.771703 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767605    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767607    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767610    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767612    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767615    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767617    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767620    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767622    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767625    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767628    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767630    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767633    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767635    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767638    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767641    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767644    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767646    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 18:16:16.772190 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:16.767648    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 18:16:16.772616 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.767653    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 18:16:16.772616 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.767769    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 18:16:16.772616 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.769774    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 18:16:16.772616 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.770816    2576 server.go:1019] "Starting client certificate rotation"
Sep 16 18:16:16.772616 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.770916    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:16:16.772616 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.770946    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 18:16:16.773729 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.773715    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:16:16.774814 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.774797    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 18:16:16.787481 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.787464    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 16 18:16:16.790518 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.790504    2576 log.go:25] "Validated CRI v1 image API"
Sep 16 18:16:16.791383 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.791366    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 18:16:16.796908 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.796889    2576 fs.go:135] Filesystem UUIDs: map[1ef0c057-c729-42ab-95c9-9ae85cbf2ae7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bec8a659-1c22-4582-a73a-5f5110b8a78c:/dev/nvme0n1p4]
Sep 16 18:16:16.796982 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.796905    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 18:16:16.802866 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.802754    2576 manager.go:216] Machine: {Timestamp:2026-09-16 18:16:16.802299209 +0000 UTC m=+0.358592512 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099572 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f5c1c6dba6e9434b922147acd7bdd SystemUUID:ec2f5c1c-6dba-6e94-34b9-22147acd7bdd BootID:c0e1cf02-1020-4d4a-be80-5ba0cb8af1c0 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d3:67:cf:ef Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d3:67:cf:ef Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:c2:9e:a1:18:6e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 18:16:16.802866 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.802855    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 18:16:16.803026 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.802992    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 18:16:16.803857 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.803840    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:16:16.806759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.806736    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 18:16:16.806888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.806763    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-142.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 18:16:16.807997 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.807987    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 18:16:16.808038 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.807999    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 18:16:16.808038 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.808007    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:16:16.808912 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.808902    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 18:16:16.809644 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.809634    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 18:16:16.813522 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.813511    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:16:16.813638 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.813628    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 18:16:16.813684 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.813659    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 18:16:16.813684 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.813668    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 18:16:16.813684 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.813685    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 18:16:16.813812 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.813693    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 16 18:16:16.813812 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.813702    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 18:16:16.814055 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.814042    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:16:16.814101 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.814067    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 18:16:16.816267 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.816252    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 18:16:16.818824 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.818811    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 18:16:16.818873 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.818840    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 18:16:16.821036 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821025    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 18:16:16.821078 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821042    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 18:16:16.821078 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821048    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 18:16:16.821078 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821054    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 18:16:16.821078 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821063    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 18:16:16.821078 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821068    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 18:16:16.821231 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821082    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 18:16:16.821231 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821087    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 18:16:16.821231 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821094    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 18:16:16.821231 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821100    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 18:16:16.821231 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821108    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 18:16:16.821231 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.821113    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 18:16:16.822574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.822562    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 18:16:16.822610 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.822575    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 18:16:16.825679 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.825658    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 18:16:16.825739 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.825711    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 18:16:16.826140 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.826106    2576 server.go:1268] "Started kubelet"
Sep 16 18:16:16.826243 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.826194    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 18:16:16.826437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.826294    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 18:16:16.826607 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.826587    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 18:16:16.826935 ip-10-0-129-142 systemd[1]: Started Kubernetes Kubelet.
Sep 16 18:16:16.828070 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.828044    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 18:16:16.830675 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.830656    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 18:16:16.831523 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.830724    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-npsnx"
Sep 16 18:16:16.832327 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.832307    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 18:16:16.835162 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.835141    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 18:16:16.835788 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.835771    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 18:16:16.835963 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.835931    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:16.836521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.836451    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 18:16:16.836521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.836453    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 18:16:16.836521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.836475    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 18:16:16.836693 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.836576    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 18:16:16.836693 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.836585    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 18:16:16.836890 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.836874    2576 factory.go:55] Registering systemd factory
Sep 16 18:16:16.836927 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.836894    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 16 18:16:16.837131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.837111    2576 factory.go:153] Registering CRI-O factory
Sep 16 18:16:16.837131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.837129    2576 factory.go:223] Registration of the crio container factory successfully
Sep 16 18:16:16.837257 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.837180    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 18:16:16.837257 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.837219    2576 factory.go:103] Registering Raw factory
Sep 16 18:16:16.837257 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.837234    2576 manager.go:1196] Started watching for new ooms in manager
Sep 16 18:16:16.837708 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.837686    2576 manager.go:349] Starting recovery of all containers
Sep 16 18:16:16.839419 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.839392    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-npsnx"
Sep 16 18:16:16.839835 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.839799    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-142.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 18:16:16.839960 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.839944    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-142.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 18:16:16.840034 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.840013    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 18:16:16.841093 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.839585    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-142.ec2.internal.18d5dfdf3bf25845  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-142.ec2.internal,UID:ip-10-0-129-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-142.ec2.internal,},FirstTimestamp:2026-09-16 18:16:16.826079301 +0000 UTC m=+0.382372602,LastTimestamp:2026-09-16 18:16:16.826079301 +0000 UTC m=+0.382372602,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-142.ec2.internal,}"
Sep 16 18:16:16.850926 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.850906    2576 manager.go:354] Recovery completed
Sep 16 18:16:16.852220 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.852183    2576 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 16 18:16:16.855639 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.855626    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:16.857233 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.857196    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:16.857293 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.857238    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:16.857293 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.857251    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:16.857751 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.857734    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 18:16:16.857751 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.857746    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:16.857868 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.857748    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 18:16:16.857868 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.857824    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 18:16:16.858721 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.858706    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:16.858780 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.858733    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:16.858780 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.858743    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:16.858963 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.858953    2576 policy_none.go:49] "None policy: Start"
Sep 16 18:16:16.859001 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.858967    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 18:16:16.859001 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.858976    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 18:16:16.859855 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.859785    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-142.ec2.internal.18d5dfdf3dcd9335  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-142.ec2.internal,UID:ip-10-0-129-142.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-142.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-142.ec2.internal,},FirstTimestamp:2026-09-16 18:16:16.857223989 +0000 UTC m=+0.413517297,LastTimestamp:2026-09-16 18:16:16.857223989 +0000 UTC m=+0.413517297,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-142.ec2.internal,}"
Sep 16 18:16:16.860104 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.860092    2576 policy_none.go:47] "Start"
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.887839    2576 manager.go:337] "Starting Device Plugin manager"
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.887869    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.887879    2576 server.go:87] "Starting device plugin registration server"
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.888113    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.888125    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.888279    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.888349    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.888358    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.889254    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 18:16:16.901193 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.889301    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:16.936564 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.936537    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 18:16:16.937751 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.937735    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 18:16:16.937827 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.937778    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 18:16:16.937827 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.937796    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 18:16:16.937921 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:16.937831    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 18:16:16.940346 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.940330    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:16.988370 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.988354    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:16.989133 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.989118    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:16.989244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.989148    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:16.989244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.989163    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:16.989244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:16.989191    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.000244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.000180    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.000244 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.000204    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-142.ec2.internal\": node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:17.021870 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.021850    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:17.038354 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.038335    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal"]
Sep 16 18:16:17.038402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.038389    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:17.039592 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.039578    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:17.039655 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.039610    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:17.039655 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.039620    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:17.040678 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.040667    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:17.040821 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.040808    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.040876 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.040836    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:17.041356 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.041338    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:17.041432 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.041367    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:17.041432 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.041378    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:17.041432 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.041343    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:17.041566 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.041450    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:17.041566 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.041468    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:17.042465 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.042451    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.042590 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.042476    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 18:16:17.043199 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.043184    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 18:16:17.043276 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.043228    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 18:16:17.043276 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.043244    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeHasSufficientPID"
Sep 16 18:16:17.074880 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.074859    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-142.ec2.internal\" not found" node="ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.079006 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.078991    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-142.ec2.internal\" not found" node="ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.122500 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.122484    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:17.138183 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.138167    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4dc07801ddc3bd80f76c3bff8bf4e7b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal\" (UID: \"4dc07801ddc3bd80f76c3bff8bf4e7b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.138271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.138187    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dc07801ddc3bd80f76c3bff8bf4e7b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal\" (UID: \"4dc07801ddc3bd80f76c3bff8bf4e7b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.138271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.138202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bca848baac81a5ff45bd401fe573fb0f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-142.ec2.internal\" (UID: \"bca848baac81a5ff45bd401fe573fb0f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.222951 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.222927    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:17.239303 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.239281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4dc07801ddc3bd80f76c3bff8bf4e7b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal\" (UID: \"4dc07801ddc3bd80f76c3bff8bf4e7b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.239385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.239311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dc07801ddc3bd80f76c3bff8bf4e7b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal\" (UID: \"4dc07801ddc3bd80f76c3bff8bf4e7b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.239385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.239337    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bca848baac81a5ff45bd401fe573fb0f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-142.ec2.internal\" (UID: \"bca848baac81a5ff45bd401fe573fb0f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.239502 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.239413    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4dc07801ddc3bd80f76c3bff8bf4e7b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal\" (UID: \"4dc07801ddc3bd80f76c3bff8bf4e7b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.239502 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.239459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4dc07801ddc3bd80f76c3bff8bf4e7b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal\" (UID: \"4dc07801ddc3bd80f76c3bff8bf4e7b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.239502 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.239480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bca848baac81a5ff45bd401fe573fb0f-config\") pod \"kube-apiserver-proxy-ip-10-0-129-142.ec2.internal\" (UID: \"bca848baac81a5ff45bd401fe573fb0f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.323722 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.323666    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:17.377100 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.377081    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.381731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.381715    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.424158 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.424018    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:17.527759 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.527731    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:17.628840 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.628776    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-142.ec2.internal\" not found"
Sep 16 18:16:17.666430 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.666409    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:17.736141 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.736119    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.748938 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.748919    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:16:17.750646 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.750631    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal"
Sep 16 18:16:17.761742 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.761726    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 18:16:17.770989 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.770967    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 18:16:17.771129 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.771106    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:16:17.771195 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.771132    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 18:16:17.814453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.814419    2576 apiserver.go:52] "Watching apiserver"
Sep 16 18:16:17.833006 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.832983    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 18:16:17.834161 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.834139    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-mnxsx","openshift-dns/node-resolver-wddn6","openshift-multus/multus-additional-cni-plugins-dt6zf","openshift-multus/network-metrics-daemon-5l5fr","openshift-network-diagnostics/network-check-target-sdc6h","kube-system/global-pull-secret-syncer-dfxvz","openshift-image-registry/node-ca-tsg74","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal","openshift-multus/multus-nns45","openshift-network-operator/iptables-alerter-zrzg9","openshift-ovn-kubernetes/ovnkube-node-6f8mp","kube-system/konnectivity-agent-hs5np","kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"]
Sep 16 18:16:17.835360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.835342    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 18:16:17.835496 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.835473    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.836493 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.836476    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.837811 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.837792    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.837947 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.837928    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-42gkn\""
Sep 16 18:16:17.838014 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.837793    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:16:17.838626 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.838569    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 18:16:17.838626 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.838600    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 18:16:17.838626 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.838600    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vzl68\""
Sep 16 18:16:17.838848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.838603    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 18:16:17.838908 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.838850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:17.838961 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.838920    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:17.839634 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.839597    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 18:16:17.839735 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.839684    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 18:16:17.839735 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.839689    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 18:16:17.839839 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.839772    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 18:16:17.840061 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.840043    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 18:16:17.840157 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.840140    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:17.840318 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.840298    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:17.840998 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.840979    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2m6ld\""
Sep 16 18:16:17.841289 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.841274    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:17.841345 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:17.841327    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:17.842307 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.842291    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:17.843435 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843419    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.843552 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843534    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7671010-0184-4ec3-ab5c-a7766d44c065-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.843602 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843562    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7671010-0184-4ec3-ab5c-a7766d44c065-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.843602 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-sysctl-conf\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.843684 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-run\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.843684 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-sys\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.843684 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843666    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6753981-9a67-4c20-a0eb-a29a092d401a-tmp\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.843803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7stws\" (UniqueName: \"kubernetes.io/projected/c6753981-9a67-4c20-a0eb-a29a092d401a-kube-api-access-7stws\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.843803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843721    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7fa44e-0a12-4f6c-9a7e-a74dab335559-tmp-dir\") pod \"node-resolver-wddn6\" (UID: \"3f7fa44e-0a12-4f6c-9a7e-a74dab335559\") " pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.843803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843742    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbwc8\" (UniqueName: \"kubernetes.io/projected/a7671010-0184-4ec3-ab5c-a7766d44c065-kube-api-access-fbwc8\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.843803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843762    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-kubernetes\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.843803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-lib-modules\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.844043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t7dv\" (UniqueName: \"kubernetes.io/projected/3f7fa44e-0a12-4f6c-9a7e-a74dab335559-kube-api-access-6t7dv\") pod \"node-resolver-wddn6\" (UID: \"3f7fa44e-0a12-4f6c-9a7e-a74dab335559\") " pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.844043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-sysconfig\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.844043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-sysctl-d\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.844043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843906    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-systemd\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.844043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-var-lib-kubelet\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.844043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3f7fa44e-0a12-4f6c-9a7e-a74dab335559-hosts-file\") pod \"node-resolver-wddn6\" (UID: \"3f7fa44e-0a12-4f6c-9a7e-a74dab335559\") " pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.844043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.843978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7671010-0184-4ec3-ab5c-a7766d44c065-system-cni-dir\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.844043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844001    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-modprobe-d\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.844043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-host\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.844437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-tuned\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.844437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844078    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7671010-0184-4ec3-ab5c-a7766d44c065-cnibin\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.844437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844095    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7671010-0184-4ec3-ab5c-a7766d44c065-cni-binary-copy\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.844437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7671010-0184-4ec3-ab5c-a7766d44c065-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.844437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844404    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 18:16:17.844437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844398    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 18:11:16 +0000 UTC" deadline="2028-05-29 05:27:16.725188102 +0000 UTC"
Sep 16 18:16:17.844437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844440    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14891h10m58.880753195s"
Sep 16 18:16:17.844732 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844410    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 18:16:17.844732 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844564    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 18:16:17.844826 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844756    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p26w7\""
Sep 16 18:16:17.844887 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.844864    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:17.845431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.845414    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 18:16:17.845524 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.845416    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vzcdn\""
Sep 16 18:16:17.846870 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.846847    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.847039 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.847018    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 18:16:17.847907 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.847883    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 18:16:17.847907 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.847893    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:16:17.848566 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.848366    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 18:16:17.849036 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.849016    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wt6kq\""
Sep 16 18:16:17.849804 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.849782    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 18:16:17.850207 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.850187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:17.850762 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.850743    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 18:16:17.852151 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.852134    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:17.854334 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.854314    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 18:16:17.854794 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.854780    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 18:16:17.854877 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.854809    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 18:16:17.855214 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.855196    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ngx8k\""
Sep 16 18:16:17.855322 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.855305    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 18:16:17.855377 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.855336    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 18:16:17.855449 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.855435    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 18:16:17.855529 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.855514    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 18:16:17.855574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.855562    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zd5ks\""
Sep 16 18:16:17.855634 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.855603    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2fpw6\""
Sep 16 18:16:17.855814 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.855797    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 18:16:17.855883 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.855867    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 18:16:17.884445 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.884399    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpzxk"
Sep 16 18:16:17.886111 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:17.886077    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc07801ddc3bd80f76c3bff8bf4e7b9.slice/crio-4084ec3d2185ca8200806a53e15ac60ce5ea54f2ca9ad72892dfe7a3bc25a555 WatchSource:0}: Error finding container 4084ec3d2185ca8200806a53e15ac60ce5ea54f2ca9ad72892dfe7a3bc25a555: Status 404 returned error can't find the container with id 4084ec3d2185ca8200806a53e15ac60ce5ea54f2ca9ad72892dfe7a3bc25a555
Sep 16 18:16:17.886374 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:17.886356    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca848baac81a5ff45bd401fe573fb0f.slice/crio-a56905f365bc3936281aa53ad801d014cc2f0a4404f686e38b308e432fbad7ca WatchSource:0}: Error finding container a56905f365bc3936281aa53ad801d014cc2f0a4404f686e38b308e432fbad7ca: Status 404 returned error can't find the container with id a56905f365bc3936281aa53ad801d014cc2f0a4404f686e38b308e432fbad7ca
Sep 16 18:16:17.893551 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.893526    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wpzxk"
Sep 16 18:16:17.894574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.894557    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:16:17.937138 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.937118    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 18:16:17.940701 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.940662    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal" event={"ID":"4dc07801ddc3bd80f76c3bff8bf4e7b9","Type":"ContainerStarted","Data":"4084ec3d2185ca8200806a53e15ac60ce5ea54f2ca9ad72892dfe7a3bc25a555"}
Sep 16 18:16:17.941855 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.941826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal" event={"ID":"bca848baac81a5ff45bd401fe573fb0f","Type":"ContainerStarted","Data":"a56905f365bc3936281aa53ad801d014cc2f0a4404f686e38b308e432fbad7ca"}
Sep 16 18:16:17.945061 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-var-lib-kubelet\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.945155 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945072    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7671010-0184-4ec3-ab5c-a7766d44c065-system-cni-dir\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.945155 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7671010-0184-4ec3-ab5c-a7766d44c065-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.945155 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945130    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/884d3137-fbc5-4c18-8573-3644ab0e7157-iptables-alerter-script\") pod \"iptables-alerter-zrzg9\" (UID: \"884d3137-fbc5-4c18-8573-3644ab0e7157\") " pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945156    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-run-netns\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945169    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a7671010-0184-4ec3-ab5c-a7766d44c065-system-cni-dir\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945179    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-kubelet\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945185    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-var-lib-kubelet\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-run-systemd\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945274    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-host\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945283    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a7671010-0184-4ec3-ab5c-a7766d44c065-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-run-netns\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.945336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-log-socket\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945352    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-host\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-cni-netd\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945431    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-sys\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6753981-9a67-4c20-a0eb-a29a092d401a-tmp\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945498    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad78a825-31ef-4ae5-9f3a-b1b3031922e5-konnectivity-ca\") pod \"konnectivity-agent-hs5np\" (UID: \"ad78a825-31ef-4ae5-9f3a-b1b3031922e5\") " pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945513    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-cni-dir\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945527    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-sys\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5dc02e3-b518-461d-aa3e-365ffd425f77-cni-binary-copy\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945564    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-socket-dir-parent\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945583    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-hostroot\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945603    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-run-ovn-kubernetes\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945619    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj4ll\" (UniqueName: \"kubernetes.io/projected/e97710c5-0726-4c75-8639-0d47ad2aea1c-kube-api-access-jj4ll\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945649    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945669    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-cnibin\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945686    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-slash\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.945836 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945701    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-var-lib-openvswitch\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945716    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-etc-openvswitch\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945731    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-run-ovn\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-registration-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-sysconfig\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945800    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-dbus\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945833    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-node-log\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945845    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-sysconfig\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-device-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7671010-0184-4ec3-ab5c-a7766d44c065-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945892    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-var-lib-kubelet\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945908    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-conf-dir\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945903    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.945926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7stws\" (UniqueName: \"kubernetes.io/projected/c6753981-9a67-4c20-a0eb-a29a092d401a-kube-api-access-7stws\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946033    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7fa44e-0a12-4f6c-9a7e-a74dab335559-tmp-dir\") pod \"node-resolver-wddn6\" (UID: \"3f7fa44e-0a12-4f6c-9a7e-a74dab335559\") " pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946137    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j99rq\" (UniqueName: \"kubernetes.io/projected/884d3137-fbc5-4c18-8573-3644ab0e7157-kube-api-access-j99rq\") pod \"iptables-alerter-zrzg9\" (UID: \"884d3137-fbc5-4c18-8573-3644ab0e7157\") " pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-cni-bin\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.946521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946186    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-socket-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad78a825-31ef-4ae5-9f3a-b1b3031922e5-agent-certs\") pod \"konnectivity-agent-hs5np\" (UID: \"ad78a825-31ef-4ae5-9f3a-b1b3031922e5\") " pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946262    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-system-cni-dir\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-var-lib-cni-multus\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946329    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-run-multus-certs\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgv2m\" (UniqueName: \"kubernetes.io/projected/b5dc02e3-b518-461d-aa3e-365ffd425f77-kube-api-access-zgv2m\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84afbba2-5b06-4cad-878b-3e67e14a733e-ovn-node-metrics-cert\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-sysctl-d\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-kubelet-config\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946471    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4b4ff85-1357-423c-b6d1-b7f7181b35ff-serviceca\") pod \"node-ca-tsg74\" (UID: \"c4b4ff85-1357-423c-b6d1-b7f7181b35ff\") " pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946498    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-sysctl-d\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946506    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946533    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kc5n\" (UniqueName: \"kubernetes.io/projected/84afbba2-5b06-4cad-878b-3e67e14a733e-kube-api-access-8kc5n\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946553    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-etc-kubernetes\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946567    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a7671010-0184-4ec3-ab5c-a7766d44c065-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwc8\" (UniqueName: \"kubernetes.io/projected/a7671010-0184-4ec3-ab5c-a7766d44c065-kube-api-access-fbwc8\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.947130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-tuned\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946641    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7671010-0184-4ec3-ab5c-a7766d44c065-cni-binary-copy\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946688    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-run-k8s-cni-cncf-io\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hcwl\" (UniqueName: \"kubernetes.io/projected/c8e23166-ac37-4b3d-8432-27ab6ae30734-kube-api-access-6hcwl\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946738    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-daemon-config\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-kubernetes\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946819    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-lib-modules\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84afbba2-5b06-4cad-878b-3e67e14a733e-env-overrides\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-sys-fs\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946900    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-kubernetes\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-systemd\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-lib-modules\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-systemd\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.946999    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f7fa44e-0a12-4f6c-9a7e-a74dab335559-tmp-dir\") pod \"node-resolver-wddn6\" (UID: \"3f7fa44e-0a12-4f6c-9a7e-a74dab335559\") " pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947155    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a7671010-0184-4ec3-ab5c-a7766d44c065-cni-binary-copy\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3f7fa44e-0a12-4f6c-9a7e-a74dab335559-hosts-file\") pod \"node-resolver-wddn6\" (UID: \"3f7fa44e-0a12-4f6c-9a7e-a74dab335559\") " pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.947683 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7671010-0184-4ec3-ab5c-a7766d44c065-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947419    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmbgq\" (UniqueName: \"kubernetes.io/projected/c4b4ff85-1357-423c-b6d1-b7f7181b35ff-kube-api-access-nmbgq\") pod \"node-ca-tsg74\" (UID: \"c4b4ff85-1357-423c-b6d1-b7f7181b35ff\") " pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947435    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-var-lib-cni-bin\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947507    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3f7fa44e-0a12-4f6c-9a7e-a74dab335559-hosts-file\") pod \"node-resolver-wddn6\" (UID: \"3f7fa44e-0a12-4f6c-9a7e-a74dab335559\") " pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-modprobe-d\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7671010-0184-4ec3-ab5c-a7766d44c065-cnibin\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947579    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a7671010-0184-4ec3-ab5c-a7766d44c065-cnibin\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947601    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/884d3137-fbc5-4c18-8573-3644ab0e7157-host-slash\") pod \"iptables-alerter-zrzg9\" (UID: \"884d3137-fbc5-4c18-8573-3644ab0e7157\") " pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-etc-selinux\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-sysctl-conf\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947636    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-modprobe-d\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-run\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947684    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-systemd-units\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947713    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t7dv\" (UniqueName: \"kubernetes.io/projected/3f7fa44e-0a12-4f6c-9a7e-a74dab335559-kube-api-access-6t7dv\") pod \"node-resolver-wddn6\" (UID: \"3f7fa44e-0a12-4f6c-9a7e-a74dab335559\") " pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4b4ff85-1357-423c-b6d1-b7f7181b35ff-host\") pod \"node-ca-tsg74\" (UID: \"c4b4ff85-1357-423c-b6d1-b7f7181b35ff\") " pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947750    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-sysctl-conf\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947760    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c6753981-9a67-4c20-a0eb-a29a092d401a-run\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.948297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947761    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-run-openvswitch\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.948825 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947804    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84afbba2-5b06-4cad-878b-3e67e14a733e-ovnkube-config\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.948825 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a7671010-0184-4ec3-ab5c-a7766d44c065-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.948825 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.947881    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84afbba2-5b06-4cad-878b-3e67e14a733e-ovnkube-script-lib\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:17.949605 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.949586    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6753981-9a67-4c20-a0eb-a29a092d401a-tmp\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.949696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.949623    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c6753981-9a67-4c20-a0eb-a29a092d401a-etc-tuned\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:17.969418 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.969383    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t7dv\" (UniqueName: \"kubernetes.io/projected/3f7fa44e-0a12-4f6c-9a7e-a74dab335559-kube-api-access-6t7dv\") pod \"node-resolver-wddn6\" (UID: \"3f7fa44e-0a12-4f6c-9a7e-a74dab335559\") " pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:17.972242 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.972196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbwc8\" (UniqueName: \"kubernetes.io/projected/a7671010-0184-4ec3-ab5c-a7766d44c065-kube-api-access-fbwc8\") pod \"multus-additional-cni-plugins-dt6zf\" (UID: \"a7671010-0184-4ec3-ab5c-a7766d44c065\") " pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:17.972358 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:17.972248    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7stws\" (UniqueName: \"kubernetes.io/projected/c6753981-9a67-4c20-a0eb-a29a092d401a-kube-api-access-7stws\") pod \"tuned-mnxsx\" (UID: \"c6753981-9a67-4c20-a0eb-a29a092d401a\") " pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:18.013828 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.013806    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:18.048603 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048577    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-systemd-units\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.048717 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4b4ff85-1357-423c-b6d1-b7f7181b35ff-host\") pod \"node-ca-tsg74\" (UID: \"c4b4ff85-1357-423c-b6d1-b7f7181b35ff\") " pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:18.048717 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048632    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-run-openvswitch\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.048717 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048655    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84afbba2-5b06-4cad-878b-3e67e14a733e-ovnkube-config\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.048717 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84afbba2-5b06-4cad-878b-3e67e14a733e-ovnkube-script-lib\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.048717 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048686    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-systemd-units\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.048717 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4b4ff85-1357-423c-b6d1-b7f7181b35ff-host\") pod \"node-ca-tsg74\" (UID: \"c4b4ff85-1357-423c-b6d1-b7f7181b35ff\") " pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048714    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-run-openvswitch\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/884d3137-fbc5-4c18-8573-3644ab0e7157-iptables-alerter-script\") pod \"iptables-alerter-zrzg9\" (UID: \"884d3137-fbc5-4c18-8573-3644ab0e7157\") " pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-run-netns\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-kubelet\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048810    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-run-systemd\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048809    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-run-netns\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048860    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-run-netns\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048864    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-run-systemd\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048862    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-kubelet\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048894    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-run-netns\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048894    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-log-socket\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-log-socket\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-cni-netd\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.048955    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.049012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.048993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad78a825-31ef-4ae5-9f3a-b1b3031922e5-konnectivity-ca\") pod \"konnectivity-agent-hs5np\" (UID: \"ad78a825-31ef-4ae5-9f3a-b1b3031922e5\") " pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.049036    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs podName:e97710c5-0726-4c75-8639-0d47ad2aea1c nodeName:}" failed. No retries permitted until 2026-09-16 18:16:18.549008003 +0000 UTC m=+2.105301318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs") pod "network-metrics-daemon-5l5fr" (UID: "e97710c5-0726-4c75-8639-0d47ad2aea1c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-cni-netd\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-cni-dir\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049087    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5dc02e3-b518-461d-aa3e-365ffd425f77-cni-binary-copy\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-socket-dir-parent\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049125    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-hostroot\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-run-ovn-kubernetes\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj4ll\" (UniqueName: \"kubernetes.io/projected/e97710c5-0726-4c75-8639-0d47ad2aea1c-kube-api-access-jj4ll\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049173    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-cnibin\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-slash\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049235    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-var-lib-openvswitch\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-etc-openvswitch\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049283    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-run-ovn\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049298    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-registration-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.049886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-dbus\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049328    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-node-log\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049343    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-device-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049358    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-var-lib-kubelet\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049372    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-conf-dir\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j99rq\" (UniqueName: \"kubernetes.io/projected/884d3137-fbc5-4c18-8573-3644ab0e7157-kube-api-access-j99rq\") pod \"iptables-alerter-zrzg9\" (UID: \"884d3137-fbc5-4c18-8573-3644ab0e7157\") " pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84afbba2-5b06-4cad-878b-3e67e14a733e-ovnkube-config\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84afbba2-5b06-4cad-878b-3e67e14a733e-ovnkube-script-lib\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049404    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-cni-bin\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049405    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/884d3137-fbc5-4c18-8573-3644ab0e7157-iptables-alerter-script\") pod \"iptables-alerter-zrzg9\" (UID: \"884d3137-fbc5-4c18-8573-3644ab0e7157\") " pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-socket-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad78a825-31ef-4ae5-9f3a-b1b3031922e5-agent-certs\") pod \"konnectivity-agent-hs5np\" (UID: \"ad78a825-31ef-4ae5-9f3a-b1b3031922e5\") " pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049453    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-system-cni-dir\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049465    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-socket-dir-parent\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049468    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-var-lib-cni-multus\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049501    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-conf-dir\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-node-log\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-run-multus-certs\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.050711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-var-lib-kubelet\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-hostroot\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ad78a825-31ef-4ae5-9f3a-b1b3031922e5-konnectivity-ca\") pod \"konnectivity-agent-hs5np\" (UID: \"ad78a825-31ef-4ae5-9f3a-b1b3031922e5\") " pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgv2m\" (UniqueName: \"kubernetes.io/projected/b5dc02e3-b518-461d-aa3e-365ffd425f77-kube-api-access-zgv2m\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049565    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-dbus\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049602    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-registration-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84afbba2-5b06-4cad-878b-3e67e14a733e-ovn-node-metrics-cert\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049633    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-var-lib-openvswitch\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049639    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-kubelet-config\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049666    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4b4ff85-1357-423c-b6d1-b7f7181b35ff-serviceca\") pod \"node-ca-tsg74\" (UID: \"c4b4ff85-1357-423c-b6d1-b7f7181b35ff\") " pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049690    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-cni-dir\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-system-cni-dir\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049467    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-run-ovn\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.049712    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049735    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-cnibin\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049744    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-cni-bin\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-slash\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5dc02e3-b518-461d-aa3e-365ffd425f77-cni-binary-copy\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-var-lib-cni-multus\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.049771    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret podName:a6598da7-8e1c-498e-be9f-ed44f6b8a43a nodeName:}" failed. No retries permitted until 2026-09-16 18:16:18.549755272 +0000 UTC m=+2.106048562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret") pod "global-pull-secret-syncer-dfxvz" (UID: "a6598da7-8e1c-498e-be9f-ed44f6b8a43a") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049747    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-etc-openvswitch\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049788    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kc5n\" (UniqueName: \"kubernetes.io/projected/84afbba2-5b06-4cad-878b-3e67e14a733e-kube-api-access-8kc5n\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049790    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049834    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84afbba2-5b06-4cad-878b-3e67e14a733e-host-run-ovn-kubernetes\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049875    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-run-multus-certs\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-device-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.049988    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-socket-dir\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050010    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-kubelet-config\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c4b4ff85-1357-423c-b6d1-b7f7181b35ff-serviceca\") pod \"node-ca-tsg74\" (UID: \"c4b4ff85-1357-423c-b6d1-b7f7181b35ff\") " pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050227    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-etc-kubernetes\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-etc-kubernetes\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050267    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-run-k8s-cni-cncf-io\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.051813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hcwl\" (UniqueName: \"kubernetes.io/projected/c8e23166-ac37-4b3d-8432-27ab6ae30734-kube-api-access-6hcwl\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050345    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-daemon-config\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-run-k8s-cni-cncf-io\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050396    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84afbba2-5b06-4cad-878b-3e67e14a733e-env-overrides\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050420    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-sys-fs\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050448    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmbgq\" (UniqueName: \"kubernetes.io/projected/c4b4ff85-1357-423c-b6d1-b7f7181b35ff-kube-api-access-nmbgq\") pod \"node-ca-tsg74\" (UID: \"c4b4ff85-1357-423c-b6d1-b7f7181b35ff\") " pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-var-lib-cni-bin\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050499    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/884d3137-fbc5-4c18-8573-3644ab0e7157-host-slash\") pod \"iptables-alerter-zrzg9\" (UID: \"884d3137-fbc5-4c18-8573-3644ab0e7157\") " pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-sys-fs\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050540    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-etc-selinux\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050563    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5dc02e3-b518-461d-aa3e-365ffd425f77-host-var-lib-cni-bin\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050596    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/884d3137-fbc5-4c18-8573-3644ab0e7157-host-slash\") pod \"iptables-alerter-zrzg9\" (UID: \"884d3137-fbc5-4c18-8573-3644ab0e7157\") " pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050673    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c8e23166-ac37-4b3d-8432-27ab6ae30734-etc-selinux\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050784    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84afbba2-5b06-4cad-878b-3e67e14a733e-env-overrides\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.050854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5dc02e3-b518-461d-aa3e-365ffd425f77-multus-daemon-config\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.052344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.052220    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ad78a825-31ef-4ae5-9f3a-b1b3031922e5-agent-certs\") pod \"konnectivity-agent-hs5np\" (UID: \"ad78a825-31ef-4ae5-9f3a-b1b3031922e5\") " pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:18.052773 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.052380    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84afbba2-5b06-4cad-878b-3e67e14a733e-ovn-node-metrics-cert\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.058295 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.058280    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:18.058341 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.058298    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:18.058341 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.058308    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sz4sg for pod openshift-network-diagnostics/network-check-target-sdc6h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:18.058402 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.058348    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg podName:a0818a01-cf7f-4ad1-a279-ec3f2737b4b1 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:18.558337987 +0000 UTC m=+2.114631290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sz4sg" (UniqueName: "kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg") pod "network-check-target-sdc6h" (UID: "a0818a01-cf7f-4ad1-a279-ec3f2737b4b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:18.061019 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.060991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kc5n\" (UniqueName: \"kubernetes.io/projected/84afbba2-5b06-4cad-878b-3e67e14a733e-kube-api-access-8kc5n\") pod \"ovnkube-node-6f8mp\" (UID: \"84afbba2-5b06-4cad-878b-3e67e14a733e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.061856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.061834    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj4ll\" (UniqueName: \"kubernetes.io/projected/e97710c5-0726-4c75-8639-0d47ad2aea1c-kube-api-access-jj4ll\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:18.062416 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.062394    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgv2m\" (UniqueName: \"kubernetes.io/projected/b5dc02e3-b518-461d-aa3e-365ffd425f77-kube-api-access-zgv2m\") pod \"multus-nns45\" (UID: \"b5dc02e3-b518-461d-aa3e-365ffd425f77\") " pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.062947 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.062931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hcwl\" (UniqueName: \"kubernetes.io/projected/c8e23166-ac37-4b3d-8432-27ab6ae30734-kube-api-access-6hcwl\") pod \"aws-ebs-csi-driver-node-gnvjj\" (UID: \"c8e23166-ac37-4b3d-8432-27ab6ae30734\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.064720 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.064693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j99rq\" (UniqueName: \"kubernetes.io/projected/884d3137-fbc5-4c18-8573-3644ab0e7157-kube-api-access-j99rq\") pod \"iptables-alerter-zrzg9\" (UID: \"884d3137-fbc5-4c18-8573-3644ab0e7157\") " pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:18.065477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.065459    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmbgq\" (UniqueName: \"kubernetes.io/projected/c4b4ff85-1357-423c-b6d1-b7f7181b35ff-kube-api-access-nmbgq\") pod \"node-ca-tsg74\" (UID: \"c4b4ff85-1357-423c-b6d1-b7f7181b35ff\") " pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:18.092286 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.092268    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:18.159705 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.159635    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mnxsx"
Sep 16 18:16:18.165282 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.165263    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wddn6"
Sep 16 18:16:18.167568 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:18.167545    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6753981_9a67_4c20_a0eb_a29a092d401a.slice/crio-44dc5a6799a365f708825dd60bfea2dfd472f2e175a30d962731a467217827e8 WatchSource:0}: Error finding container 44dc5a6799a365f708825dd60bfea2dfd472f2e175a30d962731a467217827e8: Status 404 returned error can't find the container with id 44dc5a6799a365f708825dd60bfea2dfd472f2e175a30d962731a467217827e8
Sep 16 18:16:18.172602 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:18.172582    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f7fa44e_0a12_4f6c_9a7e_a74dab335559.slice/crio-4cabc2260756548ecbad0ad0653467f8a149861033b8184eb366fec4bee3cae3 WatchSource:0}: Error finding container 4cabc2260756548ecbad0ad0653467f8a149861033b8184eb366fec4bee3cae3: Status 404 returned error can't find the container with id 4cabc2260756548ecbad0ad0653467f8a149861033b8184eb366fec4bee3cae3
Sep 16 18:16:18.189549 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.189523    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dt6zf"
Sep 16 18:16:18.194545 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.194531    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tsg74"
Sep 16 18:16:18.196469 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:18.196447    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7671010_0184_4ec3_ab5c_a7766d44c065.slice/crio-f2db16411def695eb1d21075590c7dfd4179ccfb9ff0867c24db8edc962e2f99 WatchSource:0}: Error finding container f2db16411def695eb1d21075590c7dfd4179ccfb9ff0867c24db8edc962e2f99: Status 404 returned error can't find the container with id f2db16411def695eb1d21075590c7dfd4179ccfb9ff0867c24db8edc962e2f99
Sep 16 18:16:18.201177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.201154    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nns45"
Sep 16 18:16:18.201996 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:18.201978    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b4ff85_1357_423c_b6d1_b7f7181b35ff.slice/crio-7ed342e2362ff019e819e9c31361b1c7ea0b806768fc49d0449d30806794007c WatchSource:0}: Error finding container 7ed342e2362ff019e819e9c31361b1c7ea0b806768fc49d0449d30806794007c: Status 404 returned error can't find the container with id 7ed342e2362ff019e819e9c31361b1c7ea0b806768fc49d0449d30806794007c
Sep 16 18:16:18.208120 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:18.208102    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5dc02e3_b518_461d_aa3e_365ffd425f77.slice/crio-d62f89ffec473d730a9a482122aefa437b4f84d7d46a8f181fed8bb1ca85bcb9 WatchSource:0}: Error finding container d62f89ffec473d730a9a482122aefa437b4f84d7d46a8f181fed8bb1ca85bcb9: Status 404 returned error can't find the container with id d62f89ffec473d730a9a482122aefa437b4f84d7d46a8f181fed8bb1ca85bcb9
Sep 16 18:16:18.211917 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.211899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zrzg9"
Sep 16 18:16:18.218352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.218331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:18.218552 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:18.218532    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod884d3137_fbc5_4c18_8573_3644ab0e7157.slice/crio-cc885f71175bcc809bedc772fe698f20096acfc9de0f9de7fc7f98eadc2ea0a7 WatchSource:0}: Error finding container cc885f71175bcc809bedc772fe698f20096acfc9de0f9de7fc7f98eadc2ea0a7: Status 404 returned error can't find the container with id cc885f71175bcc809bedc772fe698f20096acfc9de0f9de7fc7f98eadc2ea0a7
Sep 16 18:16:18.224499 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.224481    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:18.225056 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:18.225026    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84afbba2_5b06_4cad_878b_3e67e14a733e.slice/crio-5c608882c3020dd56db248f47616144aa6881a52ee7e5ae70b8bd8d8589d1f51 WatchSource:0}: Error finding container 5c608882c3020dd56db248f47616144aa6881a52ee7e5ae70b8bd8d8589d1f51: Status 404 returned error can't find the container with id 5c608882c3020dd56db248f47616144aa6881a52ee7e5ae70b8bd8d8589d1f51
Sep 16 18:16:18.227899 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.227879    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj"
Sep 16 18:16:18.232855 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:18.232832    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad78a825_31ef_4ae5_9f3a_b1b3031922e5.slice/crio-b876c169567b6443f9ce33d2576327cde294258f06bb33471d317d2639f48dfc WatchSource:0}: Error finding container b876c169567b6443f9ce33d2576327cde294258f06bb33471d317d2639f48dfc: Status 404 returned error can't find the container with id b876c169567b6443f9ce33d2576327cde294258f06bb33471d317d2639f48dfc
Sep 16 18:16:18.235770 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:16:18.235753    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e23166_ac37_4b3d_8432_27ab6ae30734.slice/crio-ee01f8adab2ba277a48c2871fea16086ca0bda3af8672137946d168ea792efa7 WatchSource:0}: Error finding container ee01f8adab2ba277a48c2871fea16086ca0bda3af8672137946d168ea792efa7: Status 404 returned error can't find the container with id ee01f8adab2ba277a48c2871fea16086ca0bda3af8672137946d168ea792efa7
Sep 16 18:16:18.554549 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.554327    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:18.554549 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.554380    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:18.554549 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.554523    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:18.554802 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.554591    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret podName:a6598da7-8e1c-498e-be9f-ed44f6b8a43a nodeName:}" failed. No retries permitted until 2026-09-16 18:16:19.554573 +0000 UTC m=+3.110866294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret") pod "global-pull-secret-syncer-dfxvz" (UID: "a6598da7-8e1c-498e-be9f-ed44f6b8a43a") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:18.554802 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.554610    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:18.554802 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.554672    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs podName:e97710c5-0726-4c75-8639-0d47ad2aea1c nodeName:}" failed. No retries permitted until 2026-09-16 18:16:19.554654712 +0000 UTC m=+3.110948016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs") pod "network-metrics-daemon-5l5fr" (UID: "e97710c5-0726-4c75-8639-0d47ad2aea1c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:18.635732 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.635700    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:18.655344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.655299    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:18.655511 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.655494    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:18.655575 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.655521    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:18.655575 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.655533    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sz4sg for pod openshift-network-diagnostics/network-check-target-sdc6h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:18.655671 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.655613    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg podName:a0818a01-cf7f-4ad1-a279-ec3f2737b4b1 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:19.655593138 +0000 UTC m=+3.211886443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sz4sg" (UniqueName: "kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg") pod "network-check-target-sdc6h" (UID: "a0818a01-cf7f-4ad1-a279-ec3f2737b4b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:18.896326 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.896154    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:11:17 +0000 UTC" deadline="2028-03-14 05:15:31.655144547 +0000 UTC"
Sep 16 18:16:18.896326 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.896232    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13066h59m12.758917109s"
Sep 16 18:16:18.940844 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.940332    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:18.940844 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:18.940462    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:18.974114 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.974078    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj" event={"ID":"c8e23166-ac37-4b3d-8432-27ab6ae30734","Type":"ContainerStarted","Data":"ee01f8adab2ba277a48c2871fea16086ca0bda3af8672137946d168ea792efa7"}
Sep 16 18:16:18.988112 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:18.988053    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hs5np" event={"ID":"ad78a825-31ef-4ae5-9f3a-b1b3031922e5","Type":"ContainerStarted","Data":"b876c169567b6443f9ce33d2576327cde294258f06bb33471d317d2639f48dfc"}
Sep 16 18:16:19.002665 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.002564    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zrzg9" event={"ID":"884d3137-fbc5-4c18-8573-3644ab0e7157","Type":"ContainerStarted","Data":"cc885f71175bcc809bedc772fe698f20096acfc9de0f9de7fc7f98eadc2ea0a7"}
Sep 16 18:16:19.011498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.011441    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nns45" event={"ID":"b5dc02e3-b518-461d-aa3e-365ffd425f77","Type":"ContainerStarted","Data":"d62f89ffec473d730a9a482122aefa437b4f84d7d46a8f181fed8bb1ca85bcb9"}
Sep 16 18:16:19.021182 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.021143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tsg74" event={"ID":"c4b4ff85-1357-423c-b6d1-b7f7181b35ff","Type":"ContainerStarted","Data":"7ed342e2362ff019e819e9c31361b1c7ea0b806768fc49d0449d30806794007c"}
Sep 16 18:16:19.028327 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.028298    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dt6zf" event={"ID":"a7671010-0184-4ec3-ab5c-a7766d44c065","Type":"ContainerStarted","Data":"f2db16411def695eb1d21075590c7dfd4179ccfb9ff0867c24db8edc962e2f99"}
Sep 16 18:16:19.039146 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.038316    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wddn6" event={"ID":"3f7fa44e-0a12-4f6c-9a7e-a74dab335559","Type":"ContainerStarted","Data":"4cabc2260756548ecbad0ad0653467f8a149861033b8184eb366fec4bee3cae3"}
Sep 16 18:16:19.060241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.058997    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" event={"ID":"84afbba2-5b06-4cad-878b-3e67e14a733e","Type":"ContainerStarted","Data":"5c608882c3020dd56db248f47616144aa6881a52ee7e5ae70b8bd8d8589d1f51"}
Sep 16 18:16:19.077709 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.077627    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mnxsx" event={"ID":"c6753981-9a67-4c20-a0eb-a29a092d401a","Type":"ContainerStarted","Data":"44dc5a6799a365f708825dd60bfea2dfd472f2e175a30d962731a467217827e8"}
Sep 16 18:16:19.562357 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.561634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:19.562357 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.561687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:19.562357 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.561815    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:19.562357 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.561874    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret podName:a6598da7-8e1c-498e-be9f-ed44f6b8a43a nodeName:}" failed. No retries permitted until 2026-09-16 18:16:21.561855177 +0000 UTC m=+5.118148473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret") pod "global-pull-secret-syncer-dfxvz" (UID: "a6598da7-8e1c-498e-be9f-ed44f6b8a43a") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:19.562357 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.562254    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:19.562357 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.562301    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs podName:e97710c5-0726-4c75-8639-0d47ad2aea1c nodeName:}" failed. No retries permitted until 2026-09-16 18:16:21.562287141 +0000 UTC m=+5.118580435 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs") pod "network-metrics-daemon-5l5fr" (UID: "e97710c5-0726-4c75-8639-0d47ad2aea1c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:19.662570 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.662447    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:19.662750 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.662625    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:19.662750 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.662649    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:19.662750 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.662666    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sz4sg for pod openshift-network-diagnostics/network-check-target-sdc6h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:19.662750 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.662723    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg podName:a0818a01-cf7f-4ad1-a279-ec3f2737b4b1 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:21.662706182 +0000 UTC m=+5.218999495 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sz4sg" (UniqueName: "kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg") pod "network-check-target-sdc6h" (UID: "a0818a01-cf7f-4ad1-a279-ec3f2737b4b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:19.897471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.897385    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:11:17 +0000 UTC" deadline="2028-04-17 05:57:15.519474827 +0000 UTC"
Sep 16 18:16:19.897471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.897426    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13883h40m55.622054061s"
Sep 16 18:16:19.938225 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.938189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:19.938385 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.938322    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:19.938459 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:19.938420    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:19.938629 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:19.938509    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:20.122636 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:20.122423    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 18:16:20.941397 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:20.940911    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:20.941397 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:20.941030    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:21.580913 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:21.580157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:21.580913 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:21.580245    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:21.580913 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.580404    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:21.580913 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.580462    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret podName:a6598da7-8e1c-498e-be9f-ed44f6b8a43a nodeName:}" failed. No retries permitted until 2026-09-16 18:16:25.580443006 +0000 UTC m=+9.136736296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret") pod "global-pull-secret-syncer-dfxvz" (UID: "a6598da7-8e1c-498e-be9f-ed44f6b8a43a") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:21.580913 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.580826    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:21.580913 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.580871    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs podName:e97710c5-0726-4c75-8639-0d47ad2aea1c nodeName:}" failed. No retries permitted until 2026-09-16 18:16:25.580856803 +0000 UTC m=+9.137150106 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs") pod "network-metrics-daemon-5l5fr" (UID: "e97710c5-0726-4c75-8639-0d47ad2aea1c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:21.682153 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:21.681582    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:21.682153 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.681765    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:21.682153 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.681783    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:21.682153 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.681795    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sz4sg for pod openshift-network-diagnostics/network-check-target-sdc6h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:21.682153 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.681845    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg podName:a0818a01-cf7f-4ad1-a279-ec3f2737b4b1 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:25.681828268 +0000 UTC m=+9.238121572 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sz4sg" (UniqueName: "kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg") pod "network-check-target-sdc6h" (UID: "a0818a01-cf7f-4ad1-a279-ec3f2737b4b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:21.939044 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:21.938863    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:21.939044 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.938996    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:21.939321 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:21.939246    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:21.939385 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:21.939362    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:22.938688 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:22.938655    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:22.939300 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:22.938786    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:23.938239 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:23.938187    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:23.938396 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:23.938327    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:23.938714 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:23.938696    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:23.939054 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:23.938801    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:24.938649 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:24.938616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:24.938842 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:24.938738    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:25.613175 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:25.612611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:25.613175 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:25.612653    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:25.613175 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.612756    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:25.613175 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.612807    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret podName:a6598da7-8e1c-498e-be9f-ed44f6b8a43a nodeName:}" failed. No retries permitted until 2026-09-16 18:16:33.612794388 +0000 UTC m=+17.169087677 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret") pod "global-pull-secret-syncer-dfxvz" (UID: "a6598da7-8e1c-498e-be9f-ed44f6b8a43a") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:25.613175 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.613110    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:25.613175 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.613148    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs podName:e97710c5-0726-4c75-8639-0d47ad2aea1c nodeName:}" failed. No retries permitted until 2026-09-16 18:16:33.613137891 +0000 UTC m=+17.169431180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs") pod "network-metrics-daemon-5l5fr" (UID: "e97710c5-0726-4c75-8639-0d47ad2aea1c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:25.713221 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:25.713172    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:25.713815 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.713360    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:25.713815 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.713392    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:25.713815 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.713405    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sz4sg for pod openshift-network-diagnostics/network-check-target-sdc6h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:25.713815 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.713466    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg podName:a0818a01-cf7f-4ad1-a279-ec3f2737b4b1 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:33.713448366 +0000 UTC m=+17.269741657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sz4sg" (UniqueName: "kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg") pod "network-check-target-sdc6h" (UID: "a0818a01-cf7f-4ad1-a279-ec3f2737b4b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:25.939020 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:25.938369    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:25.939020 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.938513    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:25.939020 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:25.938875    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:25.939020 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:25.938968    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:26.939494 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:26.939032    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:26.939494 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:26.939139    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:27.938257 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:27.938224    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:27.938445 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:27.938323    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:27.938445 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:27.938379    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:27.938561 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:27.938479    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:28.938144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:28.938110    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:28.938617 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:28.938249    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:29.938429 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:29.938402    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:29.938740 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:29.938519    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:29.938740 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:29.938564    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:29.938740 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:29.938633    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:30.938189 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:30.938152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:30.938387 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:30.938283    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:31.938817 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:31.938777    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:31.939270 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:31.938791    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:31.939270 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:31.938961    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:31.939270 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:31.939027    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:32.938350 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:32.938305    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:32.938517 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:32.938445    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:33.671773 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:33.671684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:33.671773 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:33.671750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:33.672269 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.671817    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:33.672269 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.671868    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:33.672269 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.671899    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs podName:e97710c5-0726-4c75-8639-0d47ad2aea1c nodeName:}" failed. No retries permitted until 2026-09-16 18:16:49.671879019 +0000 UTC m=+33.228172309 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs") pod "network-metrics-daemon-5l5fr" (UID: "e97710c5-0726-4c75-8639-0d47ad2aea1c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:33.672269 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.671916    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret podName:a6598da7-8e1c-498e-be9f-ed44f6b8a43a nodeName:}" failed. No retries permitted until 2026-09-16 18:16:49.671908676 +0000 UTC m=+33.228201967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret") pod "global-pull-secret-syncer-dfxvz" (UID: "a6598da7-8e1c-498e-be9f-ed44f6b8a43a") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:33.772059 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:33.772023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:33.772249 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.772228    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:33.772311 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.772258    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:33.772311 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.772270    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sz4sg for pod openshift-network-diagnostics/network-check-target-sdc6h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:33.772405 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.772333    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg podName:a0818a01-cf7f-4ad1-a279-ec3f2737b4b1 nodeName:}" failed. No retries permitted until 2026-09-16 18:16:49.772314571 +0000 UTC m=+33.328607878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sz4sg" (UniqueName: "kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg") pod "network-check-target-sdc6h" (UID: "a0818a01-cf7f-4ad1-a279-ec3f2737b4b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:33.938464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:33.938367    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:33.938608 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:33.938374    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:33.938608 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.938496    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:33.938707 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:33.938600    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:34.938475 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:34.938443    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:34.938921 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:34.938576    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:35.938969 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:35.938933    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:35.939438 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:35.938944    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:35.939438 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:35.939066    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:35.939438 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:35.939169    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:36.938776 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:36.938748    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:36.938949 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:36.938825    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:37.938189 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:37.938060    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:37.938189 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:37.938090    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:37.938189 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:37.938150    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:37.938533 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:37.938269    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:38.132119 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.131888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal" event={"ID":"bca848baac81a5ff45bd401fe573fb0f","Type":"ContainerStarted","Data":"049e8228da17d1bded921716e3c9672847e8c8b527c0560bb79af311a59b4856"}
Sep 16 18:16:38.134012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.133991    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" event={"ID":"84afbba2-5b06-4cad-878b-3e67e14a733e","Type":"ContainerStarted","Data":"0937631c81e83100c3522b4b9fee6452c6b11fcd96b4b71924847e60b32840ac"}
Sep 16 18:16:38.134094 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.134065    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" event={"ID":"84afbba2-5b06-4cad-878b-3e67e14a733e","Type":"ContainerStarted","Data":"40dadb9d0e1c87b5f1600e7e43dda6f346cb75ad0e9b46f252805f60e9dffede"}
Sep 16 18:16:38.134135 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.134118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" event={"ID":"84afbba2-5b06-4cad-878b-3e67e14a733e","Type":"ContainerStarted","Data":"6a62385c6221ab8186e62f405dfd71145c86fe497d8e6db283b7b62389495859"}
Sep 16 18:16:38.135464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.135442    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mnxsx" event={"ID":"c6753981-9a67-4c20-a0eb-a29a092d401a","Type":"ContainerStarted","Data":"4119160aacbe8b351dcb8c92ba99d600c895822a7a3fc091b059977685b07687"}
Sep 16 18:16:38.136764 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.136739    2576 generic.go:356] "Generic (PLEG): container finished" podID="4dc07801ddc3bd80f76c3bff8bf4e7b9" containerID="8e2f85815a62ac9859771963f1d8cc5946a76045480c1d06c624f11a8a0a9a47" exitCode=0
Sep 16 18:16:38.136850 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.136810    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal" event={"ID":"4dc07801ddc3bd80f76c3bff8bf4e7b9","Type":"ContainerDied","Data":"8e2f85815a62ac9859771963f1d8cc5946a76045480c1d06c624f11a8a0a9a47"}
Sep 16 18:16:38.138232 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.138201    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj" event={"ID":"c8e23166-ac37-4b3d-8432-27ab6ae30734","Type":"ContainerStarted","Data":"adddebb0e1a368cb7f3093012f3e40caf25b420756ed622f669bc68dbac9078e"}
Sep 16 18:16:38.139574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.139554    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hs5np" event={"ID":"ad78a825-31ef-4ae5-9f3a-b1b3031922e5","Type":"ContainerStarted","Data":"87e111e9aae3df173c289297162fdea687bb4699536ff12271e5c5f60b3fac50"}
Sep 16 18:16:38.140918 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.140890    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nns45" event={"ID":"b5dc02e3-b518-461d-aa3e-365ffd425f77","Type":"ContainerStarted","Data":"64b3a4950ebbed7001291224751e5e5d3afb356852ae4f194b19c5f926c9e872"}
Sep 16 18:16:38.142283 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.142264    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tsg74" event={"ID":"c4b4ff85-1357-423c-b6d1-b7f7181b35ff","Type":"ContainerStarted","Data":"b66d44f5e4536894727ed11e951e6beb6576ff1223b88d38b5e0be6ce14453ea"}
Sep 16 18:16:38.143758 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.143740    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7671010-0184-4ec3-ab5c-a7766d44c065" containerID="30e32476d7eb80dd1ed3679779ff9d5f70b409ff1600c44621bc82d59ca12dc0" exitCode=0
Sep 16 18:16:38.143850 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.143788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dt6zf" event={"ID":"a7671010-0184-4ec3-ab5c-a7766d44c065","Type":"ContainerDied","Data":"30e32476d7eb80dd1ed3679779ff9d5f70b409ff1600c44621bc82d59ca12dc0"}
Sep 16 18:16:38.145204 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.145181    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wddn6" event={"ID":"3f7fa44e-0a12-4f6c-9a7e-a74dab335559","Type":"ContainerStarted","Data":"5b280eb2bbea7f9a98200c39323513bbbb91dafac13d6cbecc2cfd0eb4116fc6"}
Sep 16 18:16:38.147565 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.147531    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-142.ec2.internal" podStartSLOduration=21.147520264 podStartE2EDuration="21.147520264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:38.147378959 +0000 UTC m=+21.703672273" watchObservedRunningTime="2026-09-16 18:16:38.147520264 +0000 UTC m=+21.703813596"
Sep 16 18:16:38.202166 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.202126    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tsg74" podStartSLOduration=2.105984606 podStartE2EDuration="21.202112824s" totalImagesPullingTime="19.096128218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:38.181694607 +0000 UTC m=+21.737987918" watchObservedRunningTime="2026-09-16 18:16:38.202112824 +0000 UTC m=+21.758406133"
Sep 16 18:16:38.202288 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.202261    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nns45" podStartSLOduration=2.063437498 podStartE2EDuration="21.20225458s" totalImagesPullingTime="19.138817082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:38.202111114 +0000 UTC m=+21.758404425" watchObservedRunningTime="2026-09-16 18:16:38.20225458 +0000 UTC m=+21.758547889"
Sep 16 18:16:38.237973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.237936    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-wddn6" podStartSLOduration=6.017984445 podStartE2EDuration="21.237924036s" totalImagesPullingTime="15.219939591s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:38.237517396 +0000 UTC m=+21.793810707" watchObservedRunningTime="2026-09-16 18:16:38.237924036 +0000 UTC m=+21.794217347"
Sep 16 18:16:38.257306 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.257263    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hs5np" podStartSLOduration=2.192063323 podStartE2EDuration="21.257246709s" totalImagesPullingTime="19.065183386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:38.256760102 +0000 UTC m=+21.813053414" watchObservedRunningTime="2026-09-16 18:16:38.257246709 +0000 UTC m=+21.813540021"
Sep 16 18:16:38.277639 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.277604    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mnxsx" podStartSLOduration=2.129603302 podStartE2EDuration="21.277592407s" totalImagesPullingTime="19.147989105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:38.277101051 +0000 UTC m=+21.833394361" watchObservedRunningTime="2026-09-16 18:16:38.277592407 +0000 UTC m=+21.833885718"
Sep 16 18:16:38.939387 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:38.939094    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:38.940177 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:38.939452    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:39.149725 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.149688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zrzg9" event={"ID":"884d3137-fbc5-4c18-8573-3644ab0e7157","Type":"ContainerStarted","Data":"d7e2fee0cfcbe6f83d476a779c019f3e6f47077b862f6f720509d3cce0de0dab"}
Sep 16 18:16:39.153453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.153400    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" event={"ID":"84afbba2-5b06-4cad-878b-3e67e14a733e","Type":"ContainerStarted","Data":"e77e89076bc573e90940ee1ff001d5d8021c50e5bdaf305f334f1e6cc50cb251"}
Sep 16 18:16:39.153593 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.153486    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" event={"ID":"84afbba2-5b06-4cad-878b-3e67e14a733e","Type":"ContainerStarted","Data":"1555dfa1503e1603dab302635c2024992dc35ec53d9b3e4f040ca58bd925ed18"}
Sep 16 18:16:39.153593 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.153540    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" event={"ID":"84afbba2-5b06-4cad-878b-3e67e14a733e","Type":"ContainerStarted","Data":"bf0128076746c32ce7eadfcf1f7578178c65db0176e6a2601ecb9628919ae5ef"}
Sep 16 18:16:39.155440 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.155411    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal" event={"ID":"4dc07801ddc3bd80f76c3bff8bf4e7b9","Type":"ContainerStarted","Data":"d91ab37d2a6584cae7d857264ca757b02eb4b01babdbec4fa962c141e362b07a"}
Sep 16 18:16:39.167703 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.167661    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zrzg9" podStartSLOduration=3.071926694 podStartE2EDuration="22.167639793s" totalImagesPullingTime="19.095713099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:39.167551418 +0000 UTC m=+22.723844741" watchObservedRunningTime="2026-09-16 18:16:39.167639793 +0000 UTC m=+22.723933104"
Sep 16 18:16:39.459537 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.459512    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 18:16:39.900131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.900007    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T18:16:39.459531623Z","UUID":"86caddd6-37b3-4617-aa80-347e6d2ad436","Handler":null,"Name":"","Endpoint":""}
Sep 16 18:16:39.901641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.901620    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 18:16:39.901751 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.901650    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 18:16:39.938738 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.938713    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:39.938845 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:39.938744    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:39.938845 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:39.938814    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:39.938963 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:39.938942    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:40.160243 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:40.160148    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj" event={"ID":"c8e23166-ac37-4b3d-8432-27ab6ae30734","Type":"ContainerStarted","Data":"41c14be966d4c7c21e0e18175f698a8f29b3dfd92b97e05fd4f0a0776c4f1689"}
Sep 16 18:16:40.938571 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:40.938541    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:40.938750 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:40.938676    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:41.167743 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:41.167700    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj" event={"ID":"c8e23166-ac37-4b3d-8432-27ab6ae30734","Type":"ContainerStarted","Data":"18b4c8868528067ae3cfca4676220056cee40334ad0ee1b0a3cbb9c7d8a5ca5c"}
Sep 16 18:16:41.171509 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:41.171483    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" event={"ID":"84afbba2-5b06-4cad-878b-3e67e14a733e","Type":"ContainerStarted","Data":"7199cbef1539d7173e21230ebcf5fef07db642eb583ba4d3e1df2de3ae62e57c"}
Sep 16 18:16:41.189812 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:41.189729    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-142.ec2.internal" podStartSLOduration=24.18971696 podStartE2EDuration="24.18971696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:39.187937488 +0000 UTC m=+22.744230799" watchObservedRunningTime="2026-09-16 18:16:41.18971696 +0000 UTC m=+24.746010270"
Sep 16 18:16:41.938753 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:41.938722    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:41.938958 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:41.938723    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:41.938958 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:41.938845    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:41.938958 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:41.938919    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:42.938276 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:42.938244    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:42.938642 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:42.938348    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:43.108024 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:43.107947    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:43.108541 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:43.108515    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:43.129012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:43.128965    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gnvjj" podStartSLOduration=4.613476641 podStartE2EDuration="26.12894763s" totalImagesPullingTime="21.515470989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:41.190269052 +0000 UTC m=+24.746562363" watchObservedRunningTime="2026-09-16 18:16:43.12894763 +0000 UTC m=+26.685240946"
Sep 16 18:16:43.177505 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:43.177482    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:43.177923 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:43.177902    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hs5np"
Sep 16 18:16:43.938893 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:43.938711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:43.939415 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:43.938770    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:43.939415 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:43.938962    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:43.939415 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:43.939076    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:44.181626 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:44.181592    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7671010-0184-4ec3-ab5c-a7766d44c065" containerID="65143fb68c8a9c8f74323d901ec439aef1d7a7248f23ecd63d6b218cf8a2c484" exitCode=0
Sep 16 18:16:44.181866 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:44.181677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dt6zf" event={"ID":"a7671010-0184-4ec3-ab5c-a7766d44c065","Type":"ContainerDied","Data":"65143fb68c8a9c8f74323d901ec439aef1d7a7248f23ecd63d6b218cf8a2c484"}
Sep 16 18:16:44.185646 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:44.185512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" event={"ID":"84afbba2-5b06-4cad-878b-3e67e14a733e","Type":"ContainerStarted","Data":"a3e52222b207e2d1ff168f540c067317249ab2e5d173bbf57a68b1724df6c043"}
Sep 16 18:16:44.185719 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:44.185663    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:44.273875 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:44.273793    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp" podStartSLOduration=7.762724954 podStartE2EDuration="27.27377932s" totalImagesPullingTime="19.511054366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:44.273749091 +0000 UTC m=+27.830042402" watchObservedRunningTime="2026-09-16 18:16:44.27377932 +0000 UTC m=+27.830072631"
Sep 16 18:16:44.939025 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:44.938995    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:44.939582 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:44.939117    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:45.099694 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.099624    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdc6h"]
Sep 16 18:16:45.105933 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.105902    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dfxvz"]
Sep 16 18:16:45.106074 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.106018    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:45.106146 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:45.106123    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:45.106679 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.106662    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5l5fr"]
Sep 16 18:16:45.106750 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.106740    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:45.106997 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:45.106817    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:45.187714 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.187692    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:45.187842 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:45.187787    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:45.188318 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.188284    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:45.188420 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.188361    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:45.202970 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.202948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:45.203090 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:45.203077    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:16:46.191474 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:46.191279    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7671010-0184-4ec3-ab5c-a7766d44c065" containerID="7a395d50fa853274c31be345bdbcf54c37b622d30019813e1f5ed8a1793aabf2" exitCode=0
Sep 16 18:16:46.191848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:46.191363    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dt6zf" event={"ID":"a7671010-0184-4ec3-ab5c-a7766d44c065","Type":"ContainerDied","Data":"7a395d50fa853274c31be345bdbcf54c37b622d30019813e1f5ed8a1793aabf2"}
Sep 16 18:16:46.941747 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:46.941722    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:46.941828 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:46.941759    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:46.941863 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:46.941834    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:46.941863 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:46.941841    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:46.941976 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:46.941946    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:46.942076 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:46.942051    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:47.196760 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:47.196732    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7671010-0184-4ec3-ab5c-a7766d44c065" containerID="285e32c0e7549a8ab422376064eed7ff5d1e2564ffa4f65dbb2a6d80a8a0d27e" exitCode=0
Sep 16 18:16:47.197111 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:47.196805    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dt6zf" event={"ID":"a7671010-0184-4ec3-ab5c-a7766d44c065","Type":"ContainerDied","Data":"285e32c0e7549a8ab422376064eed7ff5d1e2564ffa4f65dbb2a6d80a8a0d27e"}
Sep 16 18:16:47.753138 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:47.753106    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wddn6_3f7fa44e-0a12-4f6c-9a7e-a74dab335559/dns-node-resolver/0.log"
Sep 16 18:16:48.937040 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:48.937012    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tsg74_c4b4ff85-1357-423c-b6d1-b7f7181b35ff/node-ca/0.log"
Sep 16 18:16:48.938286 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:48.938243    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:48.938426 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:48.938242    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:48.938426 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:48.938375    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:48.938426 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:48.938248    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:48.938583 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:48.938438    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:48.938583 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:48.938467    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:49.686225 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:49.686183    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:49.686511 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:49.686255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:49.686511 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:49.686307    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:49.686511 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:49.686331    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:49.686511 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:49.686375    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs podName:e97710c5-0726-4c75-8639-0d47ad2aea1c nodeName:}" failed. No retries permitted until 2026-09-16 18:17:21.686357582 +0000 UTC m=+65.242650885 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs") pod "network-metrics-daemon-5l5fr" (UID: "e97710c5-0726-4c75-8639-0d47ad2aea1c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 18:16:49.686511 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:49.686388    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret podName:a6598da7-8e1c-498e-be9f-ed44f6b8a43a nodeName:}" failed. No retries permitted until 2026-09-16 18:17:21.686382752 +0000 UTC m=+65.242676042 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret") pod "global-pull-secret-syncer-dfxvz" (UID: "a6598da7-8e1c-498e-be9f-ed44f6b8a43a") : object "kube-system"/"original-pull-secret" not registered
Sep 16 18:16:49.787299 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:49.787259    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:49.787451 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:49.787436    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 18:16:49.787503 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:49.787458    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 18:16:49.787503 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:49.787469    2576 projected.go:196] Error preparing data for projected volume kube-api-access-sz4sg for pod openshift-network-diagnostics/network-check-target-sdc6h: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:49.787566 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:49.787516    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg podName:a0818a01-cf7f-4ad1-a279-ec3f2737b4b1 nodeName:}" failed. No retries permitted until 2026-09-16 18:17:21.787503052 +0000 UTC m=+65.343796341 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sz4sg" (UniqueName: "kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg") pod "network-check-target-sdc6h" (UID: "a0818a01-cf7f-4ad1-a279-ec3f2737b4b1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 18:16:50.938665 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:50.938633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:50.938665 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:50.938670    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:50.939135 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:50.938638    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:50.939135 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:50.938745    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:50.939135 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:50.938814    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:50.939135 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:50.938882    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:52.211348 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:52.211311    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7671010-0184-4ec3-ab5c-a7766d44c065" containerID="35d501248aeef29ebb467f075f08f3f171680f4c4ce1c1de40f43f218de98b40" exitCode=0
Sep 16 18:16:52.212051 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:52.211355    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dt6zf" event={"ID":"a7671010-0184-4ec3-ab5c-a7766d44c065","Type":"ContainerDied","Data":"35d501248aeef29ebb467f075f08f3f171680f4c4ce1c1de40f43f218de98b40"}
Sep 16 18:16:52.939252 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:52.939054    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:52.939426 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:52.939062    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:52.939426 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:52.939323    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:52.939426 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:52.939065    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:52.939426 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:52.939407    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:52.939582 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:52.939498    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:53.216291 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:53.216198    2576 generic.go:356] "Generic (PLEG): container finished" podID="a7671010-0184-4ec3-ab5c-a7766d44c065" containerID="2a1f6353f6737a73462233213727fff6b6479f255a4e8e852860cc6990dbbe4a" exitCode=0
Sep 16 18:16:53.216291 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:53.216244    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dt6zf" event={"ID":"a7671010-0184-4ec3-ab5c-a7766d44c065","Type":"ContainerDied","Data":"2a1f6353f6737a73462233213727fff6b6479f255a4e8e852860cc6990dbbe4a"}
Sep 16 18:16:54.221390 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:54.221361    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dt6zf" event={"ID":"a7671010-0184-4ec3-ab5c-a7766d44c065","Type":"ContainerStarted","Data":"c19d9285cf4908e7312006a0739544c119c35c552db3b6a5596249bcf4894d95"}
Sep 16 18:16:54.250281 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:54.250227    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dt6zf" podStartSLOduration=6.887389056 podStartE2EDuration="37.250202305s" totalImagesPullingTime="30.362813249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:16:54.250176888 +0000 UTC m=+37.806470200" watchObservedRunningTime="2026-09-16 18:16:54.250202305 +0000 UTC m=+37.806495610"
Sep 16 18:16:54.938271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:54.938239    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:54.938434 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:54.938243    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:54.938434 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:54.938348    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:54.938434 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:54.938240    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:54.938556 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:54.938432    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:54.938556 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:54.938507    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:56.939249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:56.939219    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:56.939837 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:56.939261    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:56.939837 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:56.939336    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:56.939837 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:56.939336    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:16:56.939837 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:56.939407    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:56.939837 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:56.939485    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:58.938920 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:58.938887    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:16:58.939335 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:58.938891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:16:58.939335 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:58.938989    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:16:58.939335 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:16:58.939004    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:16:58.939335 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:58.939048    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:16:58.939335 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:16:58.939169    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:17:00.939051 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:00.939021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:00.939454 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:00.939021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:00.939454 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:00.939260    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:17:00.939454 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:00.939112    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:17:00.939454 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:00.939021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:00.939454 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:00.939342    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:17:02.938428 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:02.938399    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:02.938428 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:02.938418    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:02.938807 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:02.938399    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:02.938807 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:02.938508    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:17:02.938807 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:02.938558    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:17:02.938807 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:02.938633    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:17:04.941290 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:04.941261    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:04.941677 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:04.941344    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:04.941677 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:04.941353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:04.941677 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:04.941347    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:17:04.941677 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:04.941426    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:17:04.941677 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:04.941498    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:17:06.940540 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:06.940512    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:06.941064 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:06.940512    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:06.941064 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:06.940616    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:17:06.941064 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:06.940512    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:06.941064 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:06.940674    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:17:06.941064 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:06.940744    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:17:08.940956 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:08.940922    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:08.941352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:08.941012    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:08.941352 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:08.941019    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:17:08.941352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:08.941037    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:08.941352 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:08.941096    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:17:08.941352 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:08.941156    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:17:10.940910 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:10.940885    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:10.941441 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:10.940979    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:17:10.941441 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:10.940980    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:10.941441 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:10.940998    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:10.941441 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:10.941043    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:17:10.941441 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:10.941086    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:17:12.938479 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:12.938447    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:12.938479 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:12.938483    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:12.938883 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:12.938454    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:12.938883 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:12.938548    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdc6h" podUID="a0818a01-cf7f-4ad1-a279-ec3f2737b4b1"
Sep 16 18:17:12.938883 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:12.938634    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5l5fr" podUID="e97710c5-0726-4c75-8639-0d47ad2aea1c"
Sep 16 18:17:12.938883 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:12.938715    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dfxvz" podUID="a6598da7-8e1c-498e-be9f-ed44f6b8a43a"
Sep 16 18:17:13.769708 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.769684    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-142.ec2.internal" event="NodeReady"
Sep 16 18:17:13.769869 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.769766    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 18:17:13.874912 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.874887    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mcq6w"]
Sep 16 18:17:13.907304 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.907282    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mcq6w"
Sep 16 18:17:13.913615 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.913289    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 18:17:13.913615 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.913431    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 18:17:13.919735 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.919715    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mcq6w"]
Sep 16 18:17:13.920147 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.920131    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7kjdd"]
Sep 16 18:17:13.933396 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.933375    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 18:17:13.933505 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.933374    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-76n2b\""
Sep 16 18:17:13.939426 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.939409    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:13.944045 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.944031    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 18:17:13.944172 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.944155    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pfff7\""
Sep 16 18:17:13.944259 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.944177    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 18:17:13.976099 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:13.976074    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7kjdd"]
Sep 16 18:17:14.007302 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.007279    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s779v"]
Sep 16 18:17:14.020956 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.020910    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.023944 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.023927    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 18:17:14.024789 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.024762    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 18:17:14.024864 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.024813    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 18:17:14.024928 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.024867    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-glhq5\""
Sep 16 18:17:14.024928 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.024877    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 18:17:14.045196 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.045173    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s779v"]
Sep 16 18:17:14.061731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.061706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab4e4cca-0a08-4227-8195-52c3b543efa3-metrics-tls\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.061827 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.061744    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5445c3f6-e0a4-4389-bdd7-38b1ea61c111-cert\") pod \"ingress-canary-mcq6w\" (UID: \"5445c3f6-e0a4-4389-bdd7-38b1ea61c111\") " pod="openshift-ingress-canary/ingress-canary-mcq6w"
Sep 16 18:17:14.061827 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.061760    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ab4e4cca-0a08-4227-8195-52c3b543efa3-config-volume\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.061827 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.061814    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab4e4cca-0a08-4227-8195-52c3b543efa3-tmp-dir\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.061983 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.061847    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtj8s\" (UniqueName: \"kubernetes.io/projected/ab4e4cca-0a08-4227-8195-52c3b543efa3-kube-api-access-jtj8s\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.061983 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.061941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p9zw\" (UniqueName: \"kubernetes.io/projected/5445c3f6-e0a4-4389-bdd7-38b1ea61c111-kube-api-access-6p9zw\") pod \"ingress-canary-mcq6w\" (UID: \"5445c3f6-e0a4-4389-bdd7-38b1ea61c111\") " pod="openshift-ingress-canary/ingress-canary-mcq6w"
Sep 16 18:17:14.162914 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.162893    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab4e4cca-0a08-4227-8195-52c3b543efa3-metrics-tls\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.163020 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.162927    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e835b28-5963-49b5-9127-892f33c8ec74-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.163020 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.162972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5445c3f6-e0a4-4389-bdd7-38b1ea61c111-cert\") pod \"ingress-canary-mcq6w\" (UID: \"5445c3f6-e0a4-4389-bdd7-38b1ea61c111\") " pod="openshift-ingress-canary/ingress-canary-mcq6w"
Sep 16 18:17:14.163020 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.162996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ab4e4cca-0a08-4227-8195-52c3b543efa3-config-volume\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.163169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.163023    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab4e4cca-0a08-4227-8195-52c3b543efa3-tmp-dir\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.163169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.163047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtj8s\" (UniqueName: \"kubernetes.io/projected/ab4e4cca-0a08-4227-8195-52c3b543efa3-kube-api-access-jtj8s\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.163169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.163155    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e835b28-5963-49b5-9127-892f33c8ec74-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.163349 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.163192    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1e835b28-5963-49b5-9127-892f33c8ec74-crio-socket\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.163349 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.163281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p9zw\" (UniqueName: \"kubernetes.io/projected/5445c3f6-e0a4-4389-bdd7-38b1ea61c111-kube-api-access-6p9zw\") pod \"ingress-canary-mcq6w\" (UID: \"5445c3f6-e0a4-4389-bdd7-38b1ea61c111\") " pod="openshift-ingress-canary/ingress-canary-mcq6w"
Sep 16 18:17:14.163349 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.163309    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdf8n\" (UniqueName: \"kubernetes.io/projected/1e835b28-5963-49b5-9127-892f33c8ec74-kube-api-access-mdf8n\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.163349 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.163346    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e835b28-5963-49b5-9127-892f33c8ec74-data-volume\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.163542 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.163378    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab4e4cca-0a08-4227-8195-52c3b543efa3-tmp-dir\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.163603 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.163580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ab4e4cca-0a08-4227-8195-52c3b543efa3-config-volume\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.166809 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.166785    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ab4e4cca-0a08-4227-8195-52c3b543efa3-metrics-tls\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.166888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.166822    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5445c3f6-e0a4-4389-bdd7-38b1ea61c111-cert\") pod \"ingress-canary-mcq6w\" (UID: \"5445c3f6-e0a4-4389-bdd7-38b1ea61c111\") " pod="openshift-ingress-canary/ingress-canary-mcq6w"
Sep 16 18:17:14.180130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.180113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p9zw\" (UniqueName: \"kubernetes.io/projected/5445c3f6-e0a4-4389-bdd7-38b1ea61c111-kube-api-access-6p9zw\") pod \"ingress-canary-mcq6w\" (UID: \"5445c3f6-e0a4-4389-bdd7-38b1ea61c111\") " pod="openshift-ingress-canary/ingress-canary-mcq6w"
Sep 16 18:17:14.184951 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.184932    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtj8s\" (UniqueName: \"kubernetes.io/projected/ab4e4cca-0a08-4227-8195-52c3b543efa3-kube-api-access-jtj8s\") pod \"dns-default-7kjdd\" (UID: \"ab4e4cca-0a08-4227-8195-52c3b543efa3\") " pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.216366 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.216349    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mcq6w"
Sep 16 18:17:14.247362 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.247344    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:14.265023 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.264472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e835b28-5963-49b5-9127-892f33c8ec74-data-volume\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.265023 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.264537    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e835b28-5963-49b5-9127-892f33c8ec74-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.265023 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.264603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e835b28-5963-49b5-9127-892f33c8ec74-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.265023 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.264674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1e835b28-5963-49b5-9127-892f33c8ec74-crio-socket\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.265023 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.264759    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdf8n\" (UniqueName: \"kubernetes.io/projected/1e835b28-5963-49b5-9127-892f33c8ec74-kube-api-access-mdf8n\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.265305 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.265091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1e835b28-5963-49b5-9127-892f33c8ec74-crio-socket\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.265422 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.265402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1e835b28-5963-49b5-9127-892f33c8ec74-data-volume\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.265888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.265870    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1e835b28-5963-49b5-9127-892f33c8ec74-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.270400 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.270376    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1e835b28-5963-49b5-9127-892f33c8ec74-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.295037 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.294972    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdf8n\" (UniqueName: \"kubernetes.io/projected/1e835b28-5963-49b5-9127-892f33c8ec74-kube-api-access-mdf8n\") pod \"insights-runtime-extractor-s779v\" (UID: \"1e835b28-5963-49b5-9127-892f33c8ec74\") " pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.329653 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.329626    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s779v"
Sep 16 18:17:14.566632 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.566564    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7kjdd"]
Sep 16 18:17:14.569843 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:14.569814    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4e4cca_0a08_4227_8195_52c3b543efa3.slice/crio-acc1c3bd161a5479184f0ed69700afd1a3df2a14675ba75723576f47743bf11b WatchSource:0}: Error finding container acc1c3bd161a5479184f0ed69700afd1a3df2a14675ba75723576f47743bf11b: Status 404 returned error can't find the container with id acc1c3bd161a5479184f0ed69700afd1a3df2a14675ba75723576f47743bf11b
Sep 16 18:17:14.570569 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.570545    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mcq6w"]
Sep 16 18:17:14.573501 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:14.573481    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5445c3f6_e0a4_4389_bdd7_38b1ea61c111.slice/crio-e7768d951c101906e1d92a6032638ef4dcb0d8febb409f7451d550e4b550ea1b WatchSource:0}: Error finding container e7768d951c101906e1d92a6032638ef4dcb0d8febb409f7451d550e4b550ea1b: Status 404 returned error can't find the container with id e7768d951c101906e1d92a6032638ef4dcb0d8febb409f7451d550e4b550ea1b
Sep 16 18:17:14.620958 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.620937    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s779v"]
Sep 16 18:17:14.623746 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:14.623729    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e835b28_5963_49b5_9127_892f33c8ec74.slice/crio-1779c31957550295051c182b9b7d03af380dbb10ce1eedded7186365ef952378 WatchSource:0}: Error finding container 1779c31957550295051c182b9b7d03af380dbb10ce1eedded7186365ef952378: Status 404 returned error can't find the container with id 1779c31957550295051c182b9b7d03af380dbb10ce1eedded7186365ef952378
Sep 16 18:17:14.938777 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.938749    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:14.938903 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.938749    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:14.938973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.938749    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:14.942427 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.942408    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:17:14.942819 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.942410    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:17:14.942819 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.942514    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:17:14.942819 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.942523    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gs7dj\""
Sep 16 18:17:14.942819 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.942773    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:17:14.942819 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:14.942783    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xdwg2\""
Sep 16 18:17:15.267537 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:15.267428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mcq6w" event={"ID":"5445c3f6-e0a4-4389-bdd7-38b1ea61c111","Type":"ContainerStarted","Data":"e7768d951c101906e1d92a6032638ef4dcb0d8febb409f7451d550e4b550ea1b"}
Sep 16 18:17:15.269651 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:15.269622    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s779v" event={"ID":"1e835b28-5963-49b5-9127-892f33c8ec74","Type":"ContainerStarted","Data":"13d87ed01fbf7b5689c97172ac2e49423040f211e9053faf9e3994a28e006b6c"}
Sep 16 18:17:15.269828 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:15.269677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s779v" event={"ID":"1e835b28-5963-49b5-9127-892f33c8ec74","Type":"ContainerStarted","Data":"1779c31957550295051c182b9b7d03af380dbb10ce1eedded7186365ef952378"}
Sep 16 18:17:15.271165 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:15.271132    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7kjdd" event={"ID":"ab4e4cca-0a08-4227-8195-52c3b543efa3","Type":"ContainerStarted","Data":"acc1c3bd161a5479184f0ed69700afd1a3df2a14675ba75723576f47743bf11b"}
Sep 16 18:17:16.276589 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:16.276547    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s779v" event={"ID":"1e835b28-5963-49b5-9127-892f33c8ec74","Type":"ContainerStarted","Data":"779be24098ed7b2af1135602572b843dc790286b34fcea71b80b5cc7f4bd2f71"}
Sep 16 18:17:17.208277 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.208246    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6f8mp"
Sep 16 18:17:17.289108 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.289074    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b67d87dd-2xbml"]
Sep 16 18:17:17.291377 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.291359    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.295107 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.295082    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 18:17:17.295286 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.295203    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 18:17:17.295286 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.295256    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 18:17:17.295796 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.295766    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ftnfn\""
Sep 16 18:17:17.295796 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.295788    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 18:17:17.296064 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.296044    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 18:17:17.296101 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.296072    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 18:17:17.297203 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.296984    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 18:17:17.309017 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.308996    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b67d87dd-2xbml"]
Sep 16 18:17:17.388697 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.388622    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-serving-cert\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.388918 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.388728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-oauth-config\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.388918 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.388811    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-oauth-serving-cert\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.388918 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.388838    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-service-ca\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.388918 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.388882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5s69\" (UniqueName: \"kubernetes.io/projected/275e3077-1203-4f0f-88cb-3b30d0995b31-kube-api-access-n5s69\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.389104 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.388953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-console-config\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.489438 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.489415    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-serving-cert\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.489544 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.489472    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-oauth-config\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.489544 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.489522    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-oauth-serving-cert\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.489649 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.489544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-service-ca\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.489649 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.489569    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5s69\" (UniqueName: \"kubernetes.io/projected/275e3077-1203-4f0f-88cb-3b30d0995b31-kube-api-access-n5s69\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.489649 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.489600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-console-config\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.490377 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.490354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-service-ca\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.490479 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.490356    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-console-config\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.490479 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.490358    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-oauth-serving-cert\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.492341 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.492278    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-oauth-config\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.492449 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.492428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-serving-cert\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.499993 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.499973    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5s69\" (UniqueName: \"kubernetes.io/projected/275e3077-1203-4f0f-88cb-3b30d0995b31-kube-api-access-n5s69\") pod \"console-6b67d87dd-2xbml\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") " pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.605113 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.605029    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:17.882151 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:17.882119    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b67d87dd-2xbml"]
Sep 16 18:17:17.885702 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:17.885677    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod275e3077_1203_4f0f_88cb_3b30d0995b31.slice/crio-92123028b9fffbc0f287eacec630e37281b02a2b3ba53fa02cb9d17989b7fc52 WatchSource:0}: Error finding container 92123028b9fffbc0f287eacec630e37281b02a2b3ba53fa02cb9d17989b7fc52: Status 404 returned error can't find the container with id 92123028b9fffbc0f287eacec630e37281b02a2b3ba53fa02cb9d17989b7fc52
Sep 16 18:17:18.284957 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.284917    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s779v" event={"ID":"1e835b28-5963-49b5-9127-892f33c8ec74","Type":"ContainerStarted","Data":"8e66a5279d3ff6edf0ab7f986488efb978a30f485a19602d028150cf10036a95"}
Sep 16 18:17:18.286552 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.286522    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mcq6w" event={"ID":"5445c3f6-e0a4-4389-bdd7-38b1ea61c111","Type":"ContainerStarted","Data":"ff4e080514caff43ae191a296e5964ba1fd8ec2cf08c7d31110dcfd259f24e32"}
Sep 16 18:17:18.288409 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.288380    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7kjdd" event={"ID":"ab4e4cca-0a08-4227-8195-52c3b543efa3","Type":"ContainerStarted","Data":"d7f75e080eec9c8fc377428fa6544684abde851189efec4e2383176b98fed24f"}
Sep 16 18:17:18.288557 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.288422    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7kjdd" event={"ID":"ab4e4cca-0a08-4227-8195-52c3b543efa3","Type":"ContainerStarted","Data":"4ba6d5d4b0530e5908c2e1a9f4e8abe9286ca100a93977637acab5cfe56caea2"}
Sep 16 18:17:18.288557 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.288469    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:18.289672 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.289649    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b67d87dd-2xbml" event={"ID":"275e3077-1203-4f0f-88cb-3b30d0995b31","Type":"ContainerStarted","Data":"92123028b9fffbc0f287eacec630e37281b02a2b3ba53fa02cb9d17989b7fc52"}
Sep 16 18:17:18.317379 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.317337    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s779v" podStartSLOduration=2.628039409 podStartE2EDuration="5.317322267s" totalImagesPullingTime="2.689282858s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:17:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:18.316248672 +0000 UTC m=+61.872541983" watchObservedRunningTime="2026-09-16 18:17:18.317322267 +0000 UTC m=+61.873615594"
Sep 16 18:17:18.345734 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.345681    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mcq6w" podStartSLOduration=2.489774219 podStartE2EDuration="5.345663632s" totalImagesPullingTime="2.855889413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:18.344922611 +0000 UTC m=+61.901215922" watchObservedRunningTime="2026-09-16 18:17:18.345663632 +0000 UTC m=+61.901956945"
Sep 16 18:17:18.440450 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.440389    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7kjdd" podStartSLOduration=2.581027614 podStartE2EDuration="5.440368794s" totalImagesPullingTime="2.85934118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:18.398268103 +0000 UTC m=+61.954561444" watchObservedRunningTime="2026-09-16 18:17:18.440368794 +0000 UTC m=+61.996662103"
Sep 16 18:17:18.441419 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.441392    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"]
Sep 16 18:17:18.443416 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.443396    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.455623 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.455340    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 16 18:17:18.455623 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.455352    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 18:17:18.455623 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.455418    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 16 18:17:18.455623 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.455418    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-895lm\""
Sep 16 18:17:18.455623 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.455347    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 18:17:18.455962 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.455691    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 18:17:18.487437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.487415    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"]
Sep 16 18:17:18.598405 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.598321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9cqf\" (UniqueName: \"kubernetes.io/projected/c90e00fc-eb64-424a-8431-eeb5b9a59914-kube-api-access-l9cqf\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.598405 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.598394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c90e00fc-eb64-424a-8431-eeb5b9a59914-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.598628 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.598450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c90e00fc-eb64-424a-8431-eeb5b9a59914-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.598628 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.598511    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c90e00fc-eb64-424a-8431-eeb5b9a59914-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.698871 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.698841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c90e00fc-eb64-424a-8431-eeb5b9a59914-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.699057 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.698891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c90e00fc-eb64-424a-8431-eeb5b9a59914-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.699057 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.698911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c90e00fc-eb64-424a-8431-eeb5b9a59914-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.699057 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.698968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9cqf\" (UniqueName: \"kubernetes.io/projected/c90e00fc-eb64-424a-8431-eeb5b9a59914-kube-api-access-l9cqf\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.699705 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.699678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c90e00fc-eb64-424a-8431-eeb5b9a59914-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.701740 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.701708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c90e00fc-eb64-424a-8431-eeb5b9a59914-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.701863 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.701752    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c90e00fc-eb64-424a-8431-eeb5b9a59914-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.710177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.710153    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9cqf\" (UniqueName: \"kubernetes.io/projected/c90e00fc-eb64-424a-8431-eeb5b9a59914-kube-api-access-l9cqf\") pod \"prometheus-operator-68bcd7f987-rslvr\" (UID: \"c90e00fc-eb64-424a-8431-eeb5b9a59914\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:18.754402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:18.754379    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"
Sep 16 18:17:19.065563 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:19.065532    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-rslvr"]
Sep 16 18:17:19.066198 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:19.066168    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc90e00fc_eb64_424a_8431_eeb5b9a59914.slice/crio-2022c35b1f44f1d7ad302079a4cf40f527308c020d5b860ca7aa7f1a1900628e WatchSource:0}: Error finding container 2022c35b1f44f1d7ad302079a4cf40f527308c020d5b860ca7aa7f1a1900628e: Status 404 returned error can't find the container with id 2022c35b1f44f1d7ad302079a4cf40f527308c020d5b860ca7aa7f1a1900628e
Sep 16 18:17:19.294731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:19.294687    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr" event={"ID":"c90e00fc-eb64-424a-8431-eeb5b9a59914","Type":"ContainerStarted","Data":"2022c35b1f44f1d7ad302079a4cf40f527308c020d5b860ca7aa7f1a1900628e"}
Sep 16 18:17:21.303226 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.303130    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b67d87dd-2xbml" event={"ID":"275e3077-1203-4f0f-88cb-3b30d0995b31","Type":"ContainerStarted","Data":"ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da"}
Sep 16 18:17:21.344828 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.344778    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b67d87dd-2xbml" podStartSLOduration=1.321949045 podStartE2EDuration="4.3447582s" totalImagesPullingTime="3.022809155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:21.344445541 +0000 UTC m=+64.900738853" watchObservedRunningTime="2026-09-16 18:17:21.3447582 +0000 UTC m=+64.901051512"
Sep 16 18:17:21.722643 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.722609    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:21.722813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.722659    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:21.726499 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.726481    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 18:17:21.726788 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.726770    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 18:17:21.735862 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.735843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e97710c5-0726-4c75-8639-0d47ad2aea1c-metrics-certs\") pod \"network-metrics-daemon-5l5fr\" (UID: \"e97710c5-0726-4c75-8639-0d47ad2aea1c\") " pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:21.735960 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.735930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a6598da7-8e1c-498e-be9f-ed44f6b8a43a-original-pull-secret\") pod \"global-pull-secret-syncer-dfxvz\" (UID: \"a6598da7-8e1c-498e-be9f-ed44f6b8a43a\") " pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:21.823646 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.823618    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:21.828953 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.828935    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 18:17:21.837188 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.837168    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 18:17:21.847113 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.847085    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz4sg\" (UniqueName: \"kubernetes.io/projected/a0818a01-cf7f-4ad1-a279-ec3f2737b4b1-kube-api-access-sz4sg\") pod \"network-check-target-sdc6h\" (UID: \"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1\") " pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:21.853777 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.853759    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gs7dj\""
Sep 16 18:17:21.857067 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.857049    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xdwg2\""
Sep 16 18:17:21.858801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.858783    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:21.858887 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.858817    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dfxvz"
Sep 16 18:17:21.864050 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:21.864034    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5l5fr"
Sep 16 18:17:22.182502 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.182470    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdc6h"]
Sep 16 18:17:22.186522 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:22.186499    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0818a01_cf7f_4ad1_a279_ec3f2737b4b1.slice/crio-c37ff69d4a2e564a3077001d28de54910593cdc4674888c6839ec4f76f3bceeb WatchSource:0}: Error finding container c37ff69d4a2e564a3077001d28de54910593cdc4674888c6839ec4f76f3bceeb: Status 404 returned error can't find the container with id c37ff69d4a2e564a3077001d28de54910593cdc4674888c6839ec4f76f3bceeb
Sep 16 18:17:22.209639 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.209619    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dfxvz"]
Sep 16 18:17:22.212488 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:22.212464    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6598da7_8e1c_498e_be9f_ed44f6b8a43a.slice/crio-3f9983da3dc5e41f1d8980cf967167b69790823c57f5e9e3114093186f65d5d4 WatchSource:0}: Error finding container 3f9983da3dc5e41f1d8980cf967167b69790823c57f5e9e3114093186f65d5d4: Status 404 returned error can't find the container with id 3f9983da3dc5e41f1d8980cf967167b69790823c57f5e9e3114093186f65d5d4
Sep 16 18:17:22.307369 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.307309    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5l5fr"]
Sep 16 18:17:22.307820 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.307796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dfxvz" event={"ID":"a6598da7-8e1c-498e-be9f-ed44f6b8a43a","Type":"ContainerStarted","Data":"3f9983da3dc5e41f1d8980cf967167b69790823c57f5e9e3114093186f65d5d4"}
Sep 16 18:17:22.309747 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.309722    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr" event={"ID":"c90e00fc-eb64-424a-8431-eeb5b9a59914","Type":"ContainerStarted","Data":"cfe1c456e06ca4edce1f0826bd90d65d517b9410991df188386f712aeeaf97e6"}
Sep 16 18:17:22.309864 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.309768    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr" event={"ID":"c90e00fc-eb64-424a-8431-eeb5b9a59914","Type":"ContainerStarted","Data":"13e02eb5fb29f14441fb4423be22d093b87f8721c39811956c136004d8cf192f"}
Sep 16 18:17:22.310504 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:22.310481    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode97710c5_0726_4c75_8639_0d47ad2aea1c.slice/crio-a759a99a50fb7825088f22b017c15b4e92c33beabea49f28774b8e9969b164a5 WatchSource:0}: Error finding container a759a99a50fb7825088f22b017c15b4e92c33beabea49f28774b8e9969b164a5: Status 404 returned error can't find the container with id a759a99a50fb7825088f22b017c15b4e92c33beabea49f28774b8e9969b164a5
Sep 16 18:17:22.311313 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.311284    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdc6h" event={"ID":"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1","Type":"ContainerStarted","Data":"c37ff69d4a2e564a3077001d28de54910593cdc4674888c6839ec4f76f3bceeb"}
Sep 16 18:17:22.404809 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.404759    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-rslvr" podStartSLOduration=1.475005349 podStartE2EDuration="4.404739346s" totalImagesPullingTime="2.929733997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:22.403582066 +0000 UTC m=+65.959875376" watchObservedRunningTime="2026-09-16 18:17:22.404739346 +0000 UTC m=+65.961032658"
Sep 16 18:17:22.726359 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.726326    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59475bf445-9x5hc"]
Sep 16 18:17:22.728403 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.728379    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.750113 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.750084    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 18:17:22.780814 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.780789    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59475bf445-9x5hc"]
Sep 16 18:17:22.830569 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.830537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-oauth-config\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.830696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.830581    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-service-ca\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.830696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.830613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-serving-cert\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.830696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.830652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-config\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.830696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.830677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-oauth-serving-cert\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.830913 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.830703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmcgr\" (UniqueName: \"kubernetes.io/projected/d9d5c006-a279-4b85-9394-bb409bccbdfe-kube-api-access-hmcgr\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.830913 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.830747    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-trusted-ca-bundle\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.931693 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.931665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-trusted-ca-bundle\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.931830 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.931720    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-oauth-config\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.931830 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.931749    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-service-ca\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.931830 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.931776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-serving-cert\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.931830 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.931826    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-config\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.932054 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.931850    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-oauth-serving-cert\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.932054 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.931872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmcgr\" (UniqueName: \"kubernetes.io/projected/d9d5c006-a279-4b85-9394-bb409bccbdfe-kube-api-access-hmcgr\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.932561 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.932535    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-config\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.932650 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.932628    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-trusted-ca-bundle\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.932688 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.932653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-oauth-serving-cert\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.933000 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.932957    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-service-ca\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.951062 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.938590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-serving-cert\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.951062 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.944133    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-oauth-config\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:22.958744 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:22.958330    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmcgr\" (UniqueName: \"kubernetes.io/projected/d9d5c006-a279-4b85-9394-bb409bccbdfe-kube-api-access-hmcgr\") pod \"console-59475bf445-9x5hc\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") " pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:23.040831 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:23.040760    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:23.318870 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:23.318789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5l5fr" event={"ID":"e97710c5-0726-4c75-8639-0d47ad2aea1c","Type":"ContainerStarted","Data":"a759a99a50fb7825088f22b017c15b4e92c33beabea49f28774b8e9969b164a5"}
Sep 16 18:17:23.365709 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:23.365671    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59475bf445-9x5hc"]
Sep 16 18:17:23.488313 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:23.488278    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d5c006_a279_4b85_9394_bb409bccbdfe.slice/crio-3bb53b3764d01bedfa6a65368823fa51f4dd8f273031faf67e616210300ff515 WatchSource:0}: Error finding container 3bb53b3764d01bedfa6a65368823fa51f4dd8f273031faf67e616210300ff515: Status 404 returned error can't find the container with id 3bb53b3764d01bedfa6a65368823fa51f4dd8f273031faf67e616210300ff515
Sep 16 18:17:24.324965 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:24.324873    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5l5fr" event={"ID":"e97710c5-0726-4c75-8639-0d47ad2aea1c","Type":"ContainerStarted","Data":"a60126bde9556204999619fd3b59c86c4a86a99bc8a4edfc06ab0de56b92a835"}
Sep 16 18:17:24.324965 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:24.324931    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5l5fr" event={"ID":"e97710c5-0726-4c75-8639-0d47ad2aea1c","Type":"ContainerStarted","Data":"aeff42948102d19ea1b91c41cc663cc1f7026087b3216a857029cb8047c60aab"}
Sep 16 18:17:24.326881 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:24.326848    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59475bf445-9x5hc" event={"ID":"d9d5c006-a279-4b85-9394-bb409bccbdfe","Type":"ContainerStarted","Data":"d9a4994433a9776abe266af8c289059a77b25df3ec5de466a5359822c987f276"}
Sep 16 18:17:24.327010 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:24.326897    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59475bf445-9x5hc" event={"ID":"d9d5c006-a279-4b85-9394-bb409bccbdfe","Type":"ContainerStarted","Data":"3bb53b3764d01bedfa6a65368823fa51f4dd8f273031faf67e616210300ff515"}
Sep 16 18:17:24.420589 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:24.420514    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5l5fr" podStartSLOduration=66.213932575 podStartE2EDuration="1m7.420491322s" totalImagesPullingTime="1.206558747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:24.364650659 +0000 UTC m=+67.920943970" watchObservedRunningTime="2026-09-16 18:17:24.420491322 +0000 UTC m=+67.976784634"
Sep 16 18:17:24.421131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:24.421096    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59475bf445-9x5hc" podStartSLOduration=2.421083161 podStartE2EDuration="2.421083161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:24.420443309 +0000 UTC m=+67.976736625" watchObservedRunningTime="2026-09-16 18:17:24.421083161 +0000 UTC m=+67.977376475"
Sep 16 18:17:25.530395 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.530360    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"]
Sep 16 18:17:25.532475 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.532450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.537347 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.537325    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-26lt4\""
Sep 16 18:17:25.537814 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.537798    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:17:25.545957 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.545936    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 16 18:17:25.554306 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.554282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8985fab-6493-48bf-ab68-366466f85447-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.554403 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.554324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8985fab-6493-48bf-ab68-366466f85447-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.554403 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.554349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8985fab-6493-48bf-ab68-366466f85447-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.554403 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.554377    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqnxh\" (UniqueName: \"kubernetes.io/projected/e8985fab-6493-48bf-ab68-366466f85447-kube-api-access-qqnxh\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.597025 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.596996    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"]
Sep 16 18:17:25.623147 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.623124    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"]
Sep 16 18:17:25.625156 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.625138    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.627565 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.627368    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-zqwrz\""
Sep 16 18:17:25.627565 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.627479    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 16 18:17:25.630732 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.630710    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 16 18:17:25.632299 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.632278    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 16 18:17:25.655472 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655445    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8985fab-6493-48bf-ab68-366466f85447-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.655600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8985fab-6493-48bf-ab68-366466f85447-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.655600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.655600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.655600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8985fab-6493-48bf-ab68-366466f85447-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.655600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqnxh\" (UniqueName: \"kubernetes.io/projected/e8985fab-6493-48bf-ab68-366466f85447-kube-api-access-qqnxh\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.655837 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bt7n\" (UniqueName: \"kubernetes.io/projected/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-api-access-6bt7n\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.655891 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655852    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ea1180ad-4b65-415c-973d-cd8763826bf8-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.655891 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655885    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea1180ad-4b65-415c-973d-cd8763826bf8-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.655993 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.655905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.656677 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.656653    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e8985fab-6493-48bf-ab68-366466f85447-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.658233 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.658190    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e8985fab-6493-48bf-ab68-366466f85447-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.658398 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.658377    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e8985fab-6493-48bf-ab68-366466f85447-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.691694 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.691667    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"]
Sep 16 18:17:25.755034 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.755004    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fjwrq"]
Sep 16 18:17:25.756824 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.756797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ea1180ad-4b65-415c-973d-cd8763826bf8-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.756932 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.756834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea1180ad-4b65-415c-973d-cd8763826bf8-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.756932 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.756862    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.756932 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.756890    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.756932 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.756911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.757135 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.756934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bt7n\" (UniqueName: \"kubernetes.io/projected/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-api-access-6bt7n\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.757321 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.757295    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/ea1180ad-4b65-415c-973d-cd8763826bf8-volume-directive-shadow\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.757570 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.757543    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.757643 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.757571    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea1180ad-4b65-415c-973d-cd8763826bf8-metrics-client-ca\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.757743 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.757713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.759737 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.759697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.759911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.759887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-state-metrics-tls\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.761285 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.761255    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqnxh\" (UniqueName: \"kubernetes.io/projected/e8985fab-6493-48bf-ab68-366466f85447-kube-api-access-qqnxh\") pod \"openshift-state-metrics-67dcf8c979-6wbz6\" (UID: \"e8985fab-6493-48bf-ab68-366466f85447\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.763636 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:25.763612    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"node-exporter-accelerators-collector-config\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" type="*v1.ConfigMap"
Sep 16 18:17:25.763736 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:25.763612    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-kube-rbac-proxy-config\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" type="*v1.Secret"
Sep 16 18:17:25.763736 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:25.763613    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-tls\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret"
Sep 16 18:17:25.763736 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:25.763613    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-dockercfg-976fz\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-976fz\"" type="*v1.Secret"
Sep 16 18:17:25.790755 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.790697    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bt7n\" (UniqueName: \"kubernetes.io/projected/ea1180ad-4b65-415c-973d-cd8763826bf8-kube-api-access-6bt7n\") pod \"kube-state-metrics-59fbdd77d-lcj9x\" (UID: \"ea1180ad-4b65-415c-973d-cd8763826bf8\") " pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.845606 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.845337    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"
Sep 16 18:17:25.858352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.858327    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-wtmp\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.858471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.858366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.858471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.858404    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-tls\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.858471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.858433    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-textfile\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.858471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.858461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-accelerators-collector-config\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.858680 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.858500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40510277-75cd-4c48-b079-b0408beb038c-sys\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.858680 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.858526    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm5tb\" (UniqueName: \"kubernetes.io/projected/40510277-75cd-4c48-b079-b0408beb038c-kube-api-access-jm5tb\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.858680 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.858560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40510277-75cd-4c48-b079-b0408beb038c-metrics-client-ca\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.858680 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.858589    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40510277-75cd-4c48-b079-b0408beb038c-root\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.935344 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.935314    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"
Sep 16 18:17:25.959492 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959465    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-wtmp\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959617 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959617 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959541    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-tls\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959617 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-textfile\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959617 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-accelerators-collector-config\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959841 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-wtmp\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959841 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959646    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40510277-75cd-4c48-b079-b0408beb038c-sys\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959841 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959723    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm5tb\" (UniqueName: \"kubernetes.io/projected/40510277-75cd-4c48-b079-b0408beb038c-kube-api-access-jm5tb\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959841 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959749    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40510277-75cd-4c48-b079-b0408beb038c-sys\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959841 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40510277-75cd-4c48-b079-b0408beb038c-metrics-client-ca\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.959841 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40510277-75cd-4c48-b079-b0408beb038c-root\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.960070 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959869    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/40510277-75cd-4c48-b079-b0408beb038c-root\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.960070 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.959875    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-textfile\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.960189 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.960159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40510277-75cd-4c48-b079-b0408beb038c-metrics-client-ca\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:25.993984 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:25.993965    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm5tb\" (UniqueName: \"kubernetes.io/projected/40510277-75cd-4c48-b079-b0408beb038c-kube-api-access-jm5tb\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:26.865495 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:26.865471    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 18:17:26.870245 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:26.870201    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-accelerators-collector-config\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:26.960472 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:26.960450    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-kube-rbac-proxy-config: failed to sync secret cache: timed out waiting for the condition
Sep 16 18:17:26.960633 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:26.960447    2576 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: failed to sync secret cache: timed out waiting for the condition
Sep 16 18:17:26.960633 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:26.960516    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-kube-rbac-proxy-config podName:40510277-75cd-4c48-b079-b0408beb038c nodeName:}" failed. No retries permitted until 2026-09-16 18:17:27.460499217 +0000 UTC m=+71.016792511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-kube-rbac-proxy-config" (UniqueName: "kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-kube-rbac-proxy-config") pod "node-exporter-fjwrq" (UID: "40510277-75cd-4c48-b079-b0408beb038c") : failed to sync secret cache: timed out waiting for the condition
Sep 16 18:17:26.960633 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:17:26.960591    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-tls podName:40510277-75cd-4c48-b079-b0408beb038c nodeName:}" failed. No retries permitted until 2026-09-16 18:17:27.460571167 +0000 UTC m=+71.016864456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-tls") pod "node-exporter-fjwrq" (UID: "40510277-75cd-4c48-b079-b0408beb038c") : failed to sync secret cache: timed out waiting for the condition
Sep 16 18:17:27.055890 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.055864    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x"]
Sep 16 18:17:27.057083 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:27.057060    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8985fab_6493_48bf_ab68_366466f85447.slice/crio-45ad70bb246acf70cf50a56905d8228d1b444ace74f2f6ac7cee1bca92c0fbfc WatchSource:0}: Error finding container 45ad70bb246acf70cf50a56905d8228d1b444ace74f2f6ac7cee1bca92c0fbfc: Status 404 returned error can't find the container with id 45ad70bb246acf70cf50a56905d8228d1b444ace74f2f6ac7cee1bca92c0fbfc
Sep 16 18:17:27.057266 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.057244    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6"]
Sep 16 18:17:27.062318 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:27.062297    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea1180ad_4b65_415c_973d_cd8763826bf8.slice/crio-09417e3743cc7435b08481af430eb5084c0ba500cb541e8be7b782a285a6701f WatchSource:0}: Error finding container 09417e3743cc7435b08481af430eb5084c0ba500cb541e8be7b782a285a6701f: Status 404 returned error can't find the container with id 09417e3743cc7435b08481af430eb5084c0ba500cb541e8be7b782a285a6701f
Sep 16 18:17:27.110966 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.110825    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:17:27.113149 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.113130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.136323 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.136167    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 18:17:27.136323 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.136187    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 18:17:27.136323 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.136267    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c24k4\""
Sep 16 18:17:27.136323 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.136306    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 18:17:27.136531 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.136354    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 18:17:27.136531 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.136372    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 18:17:27.136531 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.136527    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 18:17:27.136633 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.136581    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 18:17:27.136676 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.136657    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 18:17:27.143527 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.143507    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 18:17:27.169078 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.169061    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 18:17:27.172826 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.172796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.172909 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.172839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.172909 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.172883    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-volume\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.172987 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.172910    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-out\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.172987 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.172930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z47j\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-kube-api-access-9z47j\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.172987 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.172979    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.173093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.173006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.173093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.173025    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.173093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.173052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.173093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.173089    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.173249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.173129    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.173249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.173157    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-web-config\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.173249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.173194    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.191945 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.191920    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:17:27.220892 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.220865    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-976fz\""
Sep 16 18:17:27.274070 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274081    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-web-config\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274162    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-volume\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274262    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-out\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z47j\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-kube-api-access-9z47j\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274331    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274460    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274858 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.274930 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.274907    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.275759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.275433    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.277484 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.277456    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-volume\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.277882 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.277859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-web-config\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.277973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.277944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-out\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.278035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.278000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.278129 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.278109    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.278194 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.278154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.278603 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.278583    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.278914 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.278898    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.279038 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.279023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.298094 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.298076    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z47j\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-kube-api-access-9z47j\") pod \"alertmanager-main-0\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.314492 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.314441    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 18:17:27.343302 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.343269    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x" event={"ID":"ea1180ad-4b65-415c-973d-cd8763826bf8","Type":"ContainerStarted","Data":"09417e3743cc7435b08481af430eb5084c0ba500cb541e8be7b782a285a6701f"}
Sep 16 18:17:27.344801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.344779    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdc6h" event={"ID":"a0818a01-cf7f-4ad1-a279-ec3f2737b4b1","Type":"ContainerStarted","Data":"8adca51597c6791d39dd3761580161e3084c3e9376dec12d1589c1b35fccee28"}
Sep 16 18:17:27.344897 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.344856    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:27.346220 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.346188    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dfxvz" event={"ID":"a6598da7-8e1c-498e-be9f-ed44f6b8a43a","Type":"ContainerStarted","Data":"80b8279cada7f0caf7db92c0acbff30376ab20f7c39c0bbbfb07e8555c129f61"}
Sep 16 18:17:27.347957 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.347940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6" event={"ID":"e8985fab-6493-48bf-ab68-366466f85447","Type":"ContainerStarted","Data":"cc098bbae49d21067064f933b81cceba46237aaf839aeaabf73eae10d3da6382"}
Sep 16 18:17:27.348031 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.347977    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6" event={"ID":"e8985fab-6493-48bf-ab68-366466f85447","Type":"ContainerStarted","Data":"61d8b6025ab47f7d1a1ef7a2a514087411a4011b3104bfcd85f2718d6960ffd9"}
Sep 16 18:17:27.348031 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.347997    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6" event={"ID":"e8985fab-6493-48bf-ab68-366466f85447","Type":"ContainerStarted","Data":"45ad70bb246acf70cf50a56905d8228d1b444ace74f2f6ac7cee1bca92c0fbfc"}
Sep 16 18:17:27.422774 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.422751    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:17:27.436049 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.436007    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sdc6h" podStartSLOduration=65.889099062 podStartE2EDuration="1m10.435990192s" totalImagesPullingTime="4.54689113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:27.385775918 +0000 UTC m=+70.942069229" watchObservedRunningTime="2026-09-16 18:17:27.435990192 +0000 UTC m=+70.992283504"
Sep 16 18:17:27.476838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.476807    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:27.476927 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.476871    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-tls\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:27.479260 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.479235    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:27.479339 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.479258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/40510277-75cd-4c48-b079-b0408beb038c-node-exporter-tls\") pod \"node-exporter-fjwrq\" (UID: \"40510277-75cd-4c48-b079-b0408beb038c\") " pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:27.579541 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.579475    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fjwrq"
Sep 16 18:17:27.591571 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:27.591532    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40510277_75cd_4c48_b079_b0408beb038c.slice/crio-c943e36bc251cd958b08f8773618f13d2c0a4c24b7752dedea6a94d21a439103 WatchSource:0}: Error finding container c943e36bc251cd958b08f8773618f13d2c0a4c24b7752dedea6a94d21a439103: Status 404 returned error can't find the container with id c943e36bc251cd958b08f8773618f13d2c0a4c24b7752dedea6a94d21a439103
Sep 16 18:17:27.605585 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.605563    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:27.605705 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.605640    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:27.610557 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.610537    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:27.658165 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.657612    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dfxvz" podStartSLOduration=66.123530834 podStartE2EDuration="1m10.657593885s" totalImagesPullingTime="4.534063051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:16:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:27.436136126 +0000 UTC m=+70.992429434" watchObservedRunningTime="2026-09-16 18:17:27.657593885 +0000 UTC m=+71.213887201"
Sep 16 18:17:27.712714 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:27.712694    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:17:27.714471 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:27.714450    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf04ad3b7_bc0a_45de_b13e_1e754baced18.slice/crio-99730efeaaacbbd978cbfb4e0a6279940238fb55b24b987a5a4f9e30efc9701d WatchSource:0}: Error finding container 99730efeaaacbbd978cbfb4e0a6279940238fb55b24b987a5a4f9e30efc9701d: Status 404 returned error can't find the container with id 99730efeaaacbbd978cbfb4e0a6279940238fb55b24b987a5a4f9e30efc9701d
Sep 16 18:17:28.297868 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:28.297840    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7kjdd"
Sep 16 18:17:28.355356 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:28.355314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjwrq" event={"ID":"40510277-75cd-4c48-b079-b0408beb038c","Type":"ContainerStarted","Data":"c943e36bc251cd958b08f8773618f13d2c0a4c24b7752dedea6a94d21a439103"}
Sep 16 18:17:28.357481 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:28.357307    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerStarted","Data":"99730efeaaacbbd978cbfb4e0a6279940238fb55b24b987a5a4f9e30efc9701d"}
Sep 16 18:17:28.364021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:28.363996    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:17:29.365272 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.365217    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6" event={"ID":"e8985fab-6493-48bf-ab68-366466f85447","Type":"ContainerStarted","Data":"9d580c85754ab4b46b453cddab36d2500f2ae4afc901df316fea33ae3d7dc957"}
Sep 16 18:17:29.367127 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.367096    2576 generic.go:356] "Generic (PLEG): container finished" podID="40510277-75cd-4c48-b079-b0408beb038c" containerID="762e85e8375efc353a1ccd55f91cd3fe17d72aa736b9324df57f748ef8b9cc2a" exitCode=0
Sep 16 18:17:29.367271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.367177    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjwrq" event={"ID":"40510277-75cd-4c48-b079-b0408beb038c","Type":"ContainerDied","Data":"762e85e8375efc353a1ccd55f91cd3fe17d72aa736b9324df57f748ef8b9cc2a"}
Sep 16 18:17:29.370417 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.370275    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x" event={"ID":"ea1180ad-4b65-415c-973d-cd8763826bf8","Type":"ContainerStarted","Data":"8ce391dcf69c801a4d21f09beba698df4b86b2429717025ae48148f5187b7d8e"}
Sep 16 18:17:29.370417 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.370330    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x" event={"ID":"ea1180ad-4b65-415c-973d-cd8763826bf8","Type":"ContainerStarted","Data":"aaa62619df0787579c22396b1bb650e156a6944888d074a60aa3c3095114d27d"}
Sep 16 18:17:29.370417 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.370358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x" event={"ID":"ea1180ad-4b65-415c-973d-cd8763826bf8","Type":"ContainerStarted","Data":"d11eaabb59fd5df0dab3a874a782153db2e361c9f3ab25b5692c53311d63dac1"}
Sep 16 18:17:29.402082 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.401565    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-6wbz6" podStartSLOduration=2.634790366 podStartE2EDuration="4.401547672s" totalImagesPullingTime="1.766757306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:29.401338174 +0000 UTC m=+72.957631487" watchObservedRunningTime="2026-09-16 18:17:29.401547672 +0000 UTC m=+72.957841000"
Sep 16 18:17:29.538295 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.538085    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-59fbdd77d-lcj9x" podStartSLOduration=2.667118984 podStartE2EDuration="4.538065148s" totalImagesPullingTime="1.870946164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:29.466332093 +0000 UTC m=+73.022625404" watchObservedRunningTime="2026-09-16 18:17:29.538065148 +0000 UTC m=+73.094358460"
Sep 16 18:17:29.876735 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.876661    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"]
Sep 16 18:17:29.879651 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.879636    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:29.905402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.905378    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-jkr2k\""
Sep 16 18:17:29.905664 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.905641    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 16 18:17:29.926419 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:29.926394    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"]
Sep 16 18:17:30.001989 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.001957    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fglm\" (UniqueName: \"kubernetes.io/projected/c655ace3-27d3-4eab-9927-1711673bf890-kube-api-access-8fglm\") pod \"monitoring-plugin-5d85778b9d-r67dw\" (UID: \"c655ace3-27d3-4eab-9927-1711673bf890\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:30.002091 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.001998    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c655ace3-27d3-4eab-9927-1711673bf890-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-r67dw\" (UID: \"c655ace3-27d3-4eab-9927-1711673bf890\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:30.026980 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.026955    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"]
Sep 16 18:17:30.030370 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.030356    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.036105 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.036083    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 16 18:17:30.036514 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.036446    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 16 18:17:30.036603 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.036583    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-4j7th\""
Sep 16 18:17:30.036663 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.036611    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 16 18:17:30.036663 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.036634    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 16 18:17:30.037129 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.037109    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 16 18:17:30.038452 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.038431    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c52rrvtbutt9u\""
Sep 16 18:17:30.067718 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.067697    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"]
Sep 16 18:17:30.102748 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.102722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.102866 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.102802    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-grpc-tls\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.102935 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.102883    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fglm\" (UniqueName: \"kubernetes.io/projected/c655ace3-27d3-4eab-9927-1711673bf890-kube-api-access-8fglm\") pod \"monitoring-plugin-5d85778b9d-r67dw\" (UID: \"c655ace3-27d3-4eab-9927-1711673bf890\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:30.102935 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.102915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c655ace3-27d3-4eab-9927-1711673bf890-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-r67dw\" (UID: \"c655ace3-27d3-4eab-9927-1711673bf890\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:30.103043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.102943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2w6v\" (UniqueName: \"kubernetes.io/projected/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-kube-api-access-m2w6v\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.103043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.102975    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.103043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.103003    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-metrics-client-ca\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.103043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.103038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-tls\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.103272 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.103063    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.103272 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.103089    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.106067 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.106046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c655ace3-27d3-4eab-9927-1711673bf890-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-r67dw\" (UID: \"c655ace3-27d3-4eab-9927-1711673bf890\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:30.133676 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.133622    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fglm\" (UniqueName: \"kubernetes.io/projected/c655ace3-27d3-4eab-9927-1711673bf890-kube-api-access-8fglm\") pod \"monitoring-plugin-5d85778b9d-r67dw\" (UID: \"c655ace3-27d3-4eab-9927-1711673bf890\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:30.188622 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.188587    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:30.203790 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.203767    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2w6v\" (UniqueName: \"kubernetes.io/projected/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-kube-api-access-m2w6v\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.203895 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.203811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.203895 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.203840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-metrics-client-ca\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.203895 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.203873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-tls\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.204076 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.203898    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.204076 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.203922    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.204076 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.203953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.204076 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.204011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-grpc-tls\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.204739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.204684    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-metrics-client-ca\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.206987 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.206966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-tls\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.207121 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.207099    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.207166 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.207135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.207166 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.207146    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-grpc-tls\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.207372 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.207351    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.208043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.208023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.222284 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.222264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2w6v\" (UniqueName: \"kubernetes.io/projected/60a0d6a7-73b4-4c55-b257-e8549c9ac37c-kube-api-access-m2w6v\") pod \"thanos-querier-5bfd495f58-jv5bv\" (UID: \"60a0d6a7-73b4-4c55-b257-e8549c9ac37c\") " pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.339319 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.339286    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:30.377249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.377197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjwrq" event={"ID":"40510277-75cd-4c48-b079-b0408beb038c","Type":"ContainerStarted","Data":"1a7e005b462b67ce4fcaec8f2758b9fe9bcdab1147eca3675131614be66e4659"}
Sep 16 18:17:30.377646 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.377274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fjwrq" event={"ID":"40510277-75cd-4c48-b079-b0408beb038c","Type":"ContainerStarted","Data":"24fa17a26e8fb010e1986efeda72c74e23708cab061dc969bce84f19b4d95c1d"}
Sep 16 18:17:30.378875 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.378850    2576 generic.go:356] "Generic (PLEG): container finished" podID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerID="75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7" exitCode=0
Sep 16 18:17:30.378996 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.378934    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerDied","Data":"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"}
Sep 16 18:17:30.472496 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.472191    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fjwrq" podStartSLOduration=4.043584919 podStartE2EDuration="5.472168911s" totalImagesPullingTime="1.344590263s" totalInitContainerRuntime="83.993729ms" isStatefulPod=true podCreationTimestamp="2026-09-16 18:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:30.415920312 +0000 UTC m=+73.972213623" watchObservedRunningTime="2026-09-16 18:17:30.472168911 +0000 UTC m=+74.028462229"
Sep 16 18:17:30.472753 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.472725    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"]
Sep 16 18:17:30.477058 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:30.477033    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc655ace3_27d3_4eab_9927_1711673bf890.slice/crio-d28501d51a3e9f299331f01d486c869c37842807c92864ea79736bd656acddaf WatchSource:0}: Error finding container d28501d51a3e9f299331f01d486c869c37842807c92864ea79736bd656acddaf: Status 404 returned error can't find the container with id d28501d51a3e9f299331f01d486c869c37842807c92864ea79736bd656acddaf
Sep 16 18:17:30.505480 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.505461    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6876bb786b-5ssjl"]
Sep 16 18:17:30.509867 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.509849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.514674 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.514633    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 16 18:17:30.515032 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.515009    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 16 18:17:30.515108 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.515036    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 16 18:17:30.515108 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.515038    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ej5cph04kuts2\""
Sep 16 18:17:30.517857 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.517838    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-p7s8f\""
Sep 16 18:17:30.517971 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.517906    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 16 18:17:30.536311 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.536290    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6876bb786b-5ssjl"]
Sep 16 18:17:30.608150 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.608109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-metrics-server-audit-profiles\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.608150 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.608163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-secret-metrics-server-tls\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.608384 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.608182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-client-ca-bundle\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.608384 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.608323    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr8rj\" (UniqueName: \"kubernetes.io/projected/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-kube-api-access-qr8rj\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.608384 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.608373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-audit-log\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.608532 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.608416    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.608532 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.608449    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-secret-metrics-server-client-certs\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.629421 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.629402    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"]
Sep 16 18:17:30.631699 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:30.631678    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a0d6a7_73b4_4c55_b257_e8549c9ac37c.slice/crio-cdc93605158819052e2e2e41e36b360109a86b2f41e4e8496a84fbc427e2f175 WatchSource:0}: Error finding container cdc93605158819052e2e2e41e36b360109a86b2f41e4e8496a84fbc427e2f175: Status 404 returned error can't find the container with id cdc93605158819052e2e2e41e36b360109a86b2f41e4e8496a84fbc427e2f175
Sep 16 18:17:30.709707 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.709685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr8rj\" (UniqueName: \"kubernetes.io/projected/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-kube-api-access-qr8rj\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.709810 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.709716    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-audit-log\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.709810 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.709737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.709810 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.709754    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-secret-metrics-server-client-certs\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.709810 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.709774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-metrics-server-audit-profiles\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.709810 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.709797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-secret-metrics-server-tls\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.710050 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.709821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-client-ca-bundle\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.710134 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.710105    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-audit-log\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.710547 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.710526    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.710692 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.710673    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-metrics-server-audit-profiles\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.712159 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.712141    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-secret-metrics-server-client-certs\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.712728 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.712705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-secret-metrics-server-tls\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.712808 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.712744    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-client-ca-bundle\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.724119 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.724102    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr8rj\" (UniqueName: \"kubernetes.io/projected/8215605a-bdbe-4b67-9c34-2fe8e8997c0d-kube-api-access-qr8rj\") pod \"metrics-server-6876bb786b-5ssjl\" (UID: \"8215605a-bdbe-4b67-9c34-2fe8e8997c0d\") " pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:30.820528 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:30.820499    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:31.006292 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.006228    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"]
Sep 16 18:17:31.009675 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.009654    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.023798 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.023775    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 16 18:17:31.024301 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.024281    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 16 18:17:31.024449 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.024431    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 16 18:17:31.025322 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.025294    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 16 18:17:31.025412 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.025298    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9978t\""
Sep 16 18:17:31.025412 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.025351    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 16 18:17:31.031628 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.031607    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 16 18:17:31.058989 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.058965    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"]
Sep 16 18:17:31.112737 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.112707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b641af-7985-4862-8e5e-35db0bd27931-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.112908 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.112760    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-federate-client-tls\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.112908 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.112790    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12b641af-7985-4862-8e5e-35db0bd27931-metrics-client-ca\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.113035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.112934    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b641af-7985-4862-8e5e-35db0bd27931-serving-certs-ca-bundle\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.113035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.112995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-telemeter-client-tls\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.113035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.113031    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.113184 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.113071    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-secret-telemeter-client\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.113250 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.113189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dl2v\" (UniqueName: \"kubernetes.io/projected/12b641af-7985-4862-8e5e-35db0bd27931-kube-api-access-2dl2v\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.139179 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:31.139149    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8215605a_bdbe_4b67_9c34_2fe8e8997c0d.slice/crio-cba2ebb5578eb346f1460d7d639767c0494519f06c66613ef6dfe353481b81bc WatchSource:0}: Error finding container cba2ebb5578eb346f1460d7d639767c0494519f06c66613ef6dfe353481b81bc: Status 404 returned error can't find the container with id cba2ebb5578eb346f1460d7d639767c0494519f06c66613ef6dfe353481b81bc
Sep 16 18:17:31.142913 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.142890    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6876bb786b-5ssjl"]
Sep 16 18:17:31.210691 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.210653    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59475bf445-9x5hc"]
Sep 16 18:17:31.213943 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.213906    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b641af-7985-4862-8e5e-35db0bd27931-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.214051 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.213965    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-federate-client-tls\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.214051 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.213988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12b641af-7985-4862-8e5e-35db0bd27931-metrics-client-ca\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.214051 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.214027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b641af-7985-4862-8e5e-35db0bd27931-serving-certs-ca-bundle\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.214263 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.214068    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-telemeter-client-tls\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.214263 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.214100    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.214263 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.214148    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-secret-telemeter-client\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.214263 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.214192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dl2v\" (UniqueName: \"kubernetes.io/projected/12b641af-7985-4862-8e5e-35db0bd27931-kube-api-access-2dl2v\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.215309 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.214899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b641af-7985-4862-8e5e-35db0bd27931-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.215544 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.215522    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b641af-7985-4862-8e5e-35db0bd27931-serving-certs-ca-bundle\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.215658 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.215632    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/12b641af-7985-4862-8e5e-35db0bd27931-metrics-client-ca\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.217425 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.217402    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-secret-telemeter-client\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.217536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.217428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-federate-client-tls\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.217807 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.217788    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-telemeter-client-tls\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.218145 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.218129    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/12b641af-7985-4862-8e5e-35db0bd27931-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.267962 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.267895    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dl2v\" (UniqueName: \"kubernetes.io/projected/12b641af-7985-4862-8e5e-35db0bd27931-kube-api-access-2dl2v\") pod \"telemeter-client-74bcd5c6cf-vqwt2\" (UID: \"12b641af-7985-4862-8e5e-35db0bd27931\") " pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.321746 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.321694    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"
Sep 16 18:17:31.329278 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.329250    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78d5b84795-vmg5c"]
Sep 16 18:17:31.334384 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.334015    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.361496 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.361459    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78d5b84795-vmg5c"]
Sep 16 18:17:31.389118 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.389082    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl" event={"ID":"8215605a-bdbe-4b67-9c34-2fe8e8997c0d","Type":"ContainerStarted","Data":"cba2ebb5578eb346f1460d7d639767c0494519f06c66613ef6dfe353481b81bc"}
Sep 16 18:17:31.391808 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.391668    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv" event={"ID":"60a0d6a7-73b4-4c55-b257-e8549c9ac37c","Type":"ContainerStarted","Data":"cdc93605158819052e2e2e41e36b360109a86b2f41e4e8496a84fbc427e2f175"}
Sep 16 18:17:31.400247 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.399270    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw" event={"ID":"c655ace3-27d3-4eab-9927-1711673bf890","Type":"ContainerStarted","Data":"d28501d51a3e9f299331f01d486c869c37842807c92864ea79736bd656acddaf"}
Sep 16 18:17:31.415652 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.415626    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld2vj\" (UniqueName: \"kubernetes.io/projected/d6edff61-0ad4-465e-a179-32b283bd6ce4-kube-api-access-ld2vj\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.415757 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.415673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-serving-cert\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.415757 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.415702    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-config\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.415757 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.415739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-oauth-serving-cert\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.415924 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.415769    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-oauth-config\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.415924 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.415791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-service-ca\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.415924 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.415835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-trusted-ca-bundle\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.516942 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.516906    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-trusted-ca-bundle\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.517096 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.517050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld2vj\" (UniqueName: \"kubernetes.io/projected/d6edff61-0ad4-465e-a179-32b283bd6ce4-kube-api-access-ld2vj\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.517096 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.517093    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-serving-cert\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.517257 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.517122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-config\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.517257 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.517177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-oauth-serving-cert\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.517371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.517260    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-oauth-config\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.517371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.517286    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-service-ca\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.518973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.518026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-config\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.518973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.518856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-trusted-ca-bundle\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.518973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.518878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-oauth-serving-cert\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.519190 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.519077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-service-ca\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.520645 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.520619    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-serving-cert\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.521072 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.521033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-oauth-config\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.532331 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.532308    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld2vj\" (UniqueName: \"kubernetes.io/projected/d6edff61-0ad4-465e-a179-32b283bd6ce4-kube-api-access-ld2vj\") pod \"console-78d5b84795-vmg5c\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") " pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.646797 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.646765    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:31.673895 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:31.673867    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2"]
Sep 16 18:17:32.023576 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:32.023541    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b641af_7985_4862_8e5e_35db0bd27931.slice/crio-de48c8d7ce3493b12043f7979784501f808db3a1d9aac2ddaa2b0cdb9c2d6d3f WatchSource:0}: Error finding container de48c8d7ce3493b12043f7979784501f808db3a1d9aac2ddaa2b0cdb9c2d6d3f: Status 404 returned error can't find the container with id de48c8d7ce3493b12043f7979784501f808db3a1d9aac2ddaa2b0cdb9c2d6d3f
Sep 16 18:17:32.407249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:32.407093    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2" event={"ID":"12b641af-7985-4862-8e5e-35db0bd27931","Type":"ContainerStarted","Data":"de48c8d7ce3493b12043f7979784501f808db3a1d9aac2ddaa2b0cdb9c2d6d3f"}
Sep 16 18:17:33.041656 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:33.041616    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:33.337831 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:33.337756    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78d5b84795-vmg5c"]
Sep 16 18:17:33.480897 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:33.480866    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6edff61_0ad4_465e_a179_32b283bd6ce4.slice/crio-b7784706a23b0da8920885024ca762e0c97d8c56a28137f7343613644ac64776 WatchSource:0}: Error finding container b7784706a23b0da8920885024ca762e0c97d8c56a28137f7343613644ac64776: Status 404 returned error can't find the container with id b7784706a23b0da8920885024ca762e0c97d8c56a28137f7343613644ac64776
Sep 16 18:17:34.418824 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.418789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv" event={"ID":"60a0d6a7-73b4-4c55-b257-e8549c9ac37c","Type":"ContainerStarted","Data":"9c9a668302456cdce873f6014343d28b7407daafe0082c96e0fbbaddc81d7f95"}
Sep 16 18:17:34.418934 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.418910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv" event={"ID":"60a0d6a7-73b4-4c55-b257-e8549c9ac37c","Type":"ContainerStarted","Data":"5f90d3597aedc0a8e430cd2e7bd45e613f0800bcf8958d05efef1e7e8cdb461f"}
Sep 16 18:17:34.418974 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.418951    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv" event={"ID":"60a0d6a7-73b4-4c55-b257-e8549c9ac37c","Type":"ContainerStarted","Data":"fc9a2c1c0c4011541d04ff8232cc6abb4c0d0ce83472e7d300af0f29f4e023ee"}
Sep 16 18:17:34.420614 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.420588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d5b84795-vmg5c" event={"ID":"d6edff61-0ad4-465e-a179-32b283bd6ce4","Type":"ContainerStarted","Data":"9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72"}
Sep 16 18:17:34.420693 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.420635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d5b84795-vmg5c" event={"ID":"d6edff61-0ad4-465e-a179-32b283bd6ce4","Type":"ContainerStarted","Data":"b7784706a23b0da8920885024ca762e0c97d8c56a28137f7343613644ac64776"}
Sep 16 18:17:34.424177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.424151    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerStarted","Data":"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"}
Sep 16 18:17:34.424281 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.424223    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerStarted","Data":"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"}
Sep 16 18:17:34.424281 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.424259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerStarted","Data":"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"}
Sep 16 18:17:34.424379 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.424283    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerStarted","Data":"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"}
Sep 16 18:17:34.424379 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.424306    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerStarted","Data":"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"}
Sep 16 18:17:34.425936 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.425908    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw" event={"ID":"c655ace3-27d3-4eab-9927-1711673bf890","Type":"ContainerStarted","Data":"9f433d8c7632b1f63f805d101e3818be93506c714463832dfff4df71ddd830a4"}
Sep 16 18:17:34.426073 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.426056    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:34.428872 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.428794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2" event={"ID":"12b641af-7985-4862-8e5e-35db0bd27931","Type":"ContainerStarted","Data":"b99f3af2465f5e5fc9c845308c5869c76d295d156c805e0a2ce75a93d7a0cf69"}
Sep 16 18:17:34.428872 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.428844    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2" event={"ID":"12b641af-7985-4862-8e5e-35db0bd27931","Type":"ContainerStarted","Data":"e25e33a504b06040aa67d4b71cb093dd4f5f546365d2ad0f0396a74c6ed8dc9f"}
Sep 16 18:17:34.429046 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.428875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2" event={"ID":"12b641af-7985-4862-8e5e-35db0bd27931","Type":"ContainerStarted","Data":"0f20955a0d0f9ab3df7b56a98fbb6d4fc579a4e57097fa531e904ba3e7b59865"}
Sep 16 18:17:34.430397 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.430378    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl" event={"ID":"8215605a-bdbe-4b67-9c34-2fe8e8997c0d","Type":"ContainerStarted","Data":"bbda88feeb99166e43114d57af4be88705949711afb1554a27bab377da606d54"}
Sep 16 18:17:34.431630 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.431613    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw"
Sep 16 18:17:34.451962 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.451923    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78d5b84795-vmg5c" podStartSLOduration=3.451909811 podStartE2EDuration="3.451909811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:34.451293216 +0000 UTC m=+78.007586548" watchObservedRunningTime="2026-09-16 18:17:34.451909811 +0000 UTC m=+78.008203112"
Sep 16 18:17:34.529950 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.529908    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-74bcd5c6cf-vqwt2" podStartSLOduration=2.332798404 podStartE2EDuration="4.529894145s" totalImagesPullingTime="2.197095741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:34.529560068 +0000 UTC m=+78.085853392" watchObservedRunningTime="2026-09-16 18:17:34.529894145 +0000 UTC m=+78.086187456"
Sep 16 18:17:34.530847 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.530816    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-r67dw" podStartSLOduration=2.496875301 podStartE2EDuration="5.530808765s" totalImagesPullingTime="3.033933464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:34.48845714 +0000 UTC m=+78.044750451" watchObservedRunningTime="2026-09-16 18:17:34.530808765 +0000 UTC m=+78.087102135"
Sep 16 18:17:34.578087 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.578044    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl" podStartSLOduration=2.197789681 podStartE2EDuration="4.578027663s" totalImagesPullingTime="2.380237982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:34.577164312 +0000 UTC m=+78.133457634" watchObservedRunningTime="2026-09-16 18:17:34.578027663 +0000 UTC m=+78.134320977"
Sep 16 18:17:34.707822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.707736    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78d5b84795-vmg5c"]
Sep 16 18:17:34.757646 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.757618    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59dcd55c64-nt9zc"]
Sep 16 18:17:34.761033 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.761017    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.777442 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.777420    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59dcd55c64-nt9zc"]
Sep 16 18:17:34.850316 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.850288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-oauth-config\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.850418 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.850322    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-trusted-ca-bundle\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.850418 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.850348    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-oauth-serving-cert\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.850418 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.850407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-service-ca\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.850541 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.850446    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnbct\" (UniqueName: \"kubernetes.io/projected/338c680b-fe77-44e7-8c7a-868682f99f49-kube-api-access-vnbct\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.850541 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.850474    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-serving-cert\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.850541 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.850528    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-console-config\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.950866 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.950841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-console-config\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.950962 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.950896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-oauth-config\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.950962 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.950925    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-trusted-ca-bundle\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.950962 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.950953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-oauth-serving-cert\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.951081 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.950975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-service-ca\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.951282 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.951257    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnbct\" (UniqueName: \"kubernetes.io/projected/338c680b-fe77-44e7-8c7a-868682f99f49-kube-api-access-vnbct\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.951350 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.951320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-serving-cert\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.951732 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.951706    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-service-ca\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.951831 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.951706    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-console-config\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.951893 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.951824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-oauth-serving-cert\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.951976 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.951950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-trusted-ca-bundle\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.953700 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.953685    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-oauth-config\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.953812 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.953796    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-serving-cert\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:34.961318 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:34.961264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnbct\" (UniqueName: \"kubernetes.io/projected/338c680b-fe77-44e7-8c7a-868682f99f49-kube-api-access-vnbct\") pod \"console-59dcd55c64-nt9zc\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") " pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:35.070020 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:35.069993    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:35.352716 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:35.352609    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59dcd55c64-nt9zc"]
Sep 16 18:17:35.442063 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:17:35.442035    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338c680b_fe77_44e7_8c7a_868682f99f49.slice/crio-1d55beb66c00d0e2b4f76edbb01f666c17a85156b227fb81b37911ed6bf37f57 WatchSource:0}: Error finding container 1d55beb66c00d0e2b4f76edbb01f666c17a85156b227fb81b37911ed6bf37f57: Status 404 returned error can't find the container with id 1d55beb66c00d0e2b4f76edbb01f666c17a85156b227fb81b37911ed6bf37f57
Sep 16 18:17:36.441791 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.441760    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerStarted","Data":"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"}
Sep 16 18:17:36.444566 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.444543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv" event={"ID":"60a0d6a7-73b4-4c55-b257-e8549c9ac37c","Type":"ContainerStarted","Data":"5cbc2027044845b7c0e8e395c187042fca0363655270677c629ae7100365d691"}
Sep 16 18:17:36.444696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.444597    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv" event={"ID":"60a0d6a7-73b4-4c55-b257-e8549c9ac37c","Type":"ContainerStarted","Data":"35bfa9a2c3cece8ed83bc67d5c84615e864ff3b184fc7f29494bf25b5197e4ca"}
Sep 16 18:17:36.444696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.444633    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:36.444696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.444659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv" event={"ID":"60a0d6a7-73b4-4c55-b257-e8549c9ac37c","Type":"ContainerStarted","Data":"38be4eb96dbc6437a9145e82625c6c1f3d3d2da33019074e847b968625c40f89"}
Sep 16 18:17:36.446152 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.446129    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dcd55c64-nt9zc" event={"ID":"338c680b-fe77-44e7-8c7a-868682f99f49","Type":"ContainerStarted","Data":"5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87"}
Sep 16 18:17:36.446230 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.446170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dcd55c64-nt9zc" event={"ID":"338c680b-fe77-44e7-8c7a-868682f99f49","Type":"ContainerStarted","Data":"1d55beb66c00d0e2b4f76edbb01f666c17a85156b227fb81b37911ed6bf37f57"}
Sep 16 18:17:36.505600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.505534    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.42379456 podStartE2EDuration="9.505517062s" totalImagesPullingTime="6.062799298s" totalInitContainerRuntime="18.923204ms" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:36.504297145 +0000 UTC m=+80.060590456" watchObservedRunningTime="2026-09-16 18:17:36.505517062 +0000 UTC m=+80.061810375"
Sep 16 18:17:36.584233 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.584163    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv" podStartSLOduration=3.435816283 podStartE2EDuration="7.584148417s" totalImagesPullingTime="4.148332134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:36.582252474 +0000 UTC m=+80.138545781" watchObservedRunningTime="2026-09-16 18:17:36.584148417 +0000 UTC m=+80.140441728"
Sep 16 18:17:36.607774 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:36.607704    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59dcd55c64-nt9zc" podStartSLOduration=2.6076894900000003 podStartE2EDuration="2.60768949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:17:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:17:36.607200734 +0000 UTC m=+80.163494050" watchObservedRunningTime="2026-09-16 18:17:36.60768949 +0000 UTC m=+80.163982862"
Sep 16 18:17:41.647902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:41.647787    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:17:42.457142 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:42.457114    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5bfd495f58-jv5bv"
Sep 16 18:17:45.070561 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:45.070515    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:45.070561 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:45.070579    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:45.075101 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:45.075077    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:45.485268 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:45.485241    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:17:45.548567 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:45.548539    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b67d87dd-2xbml"]
Sep 16 18:17:50.821240 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:50.821193    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:50.821702 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:50.821292    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:17:56.236625 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.236568    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59475bf445-9x5hc" podUID="d9d5c006-a279-4b85-9394-bb409bccbdfe" containerName="console" containerID="cri-o://d9a4994433a9776abe266af8c289059a77b25df3ec5de466a5359822c987f276" gracePeriod=15
Sep 16 18:17:56.523513 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.523480    2576 generic.go:356] "Generic (PLEG): container finished" podID="d9d5c006-a279-4b85-9394-bb409bccbdfe" containerID="d9a4994433a9776abe266af8c289059a77b25df3ec5de466a5359822c987f276" exitCode=0
Sep 16 18:17:56.523633 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.523535    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59475bf445-9x5hc" event={"ID":"d9d5c006-a279-4b85-9394-bb409bccbdfe","Type":"ContainerDied","Data":"d9a4994433a9776abe266af8c289059a77b25df3ec5de466a5359822c987f276"}
Sep 16 18:17:56.523633 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.523592    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59475bf445-9x5hc" event={"ID":"d9d5c006-a279-4b85-9394-bb409bccbdfe","Type":"ContainerDied","Data":"3bb53b3764d01bedfa6a65368823fa51f4dd8f273031faf67e616210300ff515"}
Sep 16 18:17:56.523633 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.523606    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bb53b3764d01bedfa6a65368823fa51f4dd8f273031faf67e616210300ff515"
Sep 16 18:17:56.524253 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.524239    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:56.629170 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629142    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmcgr\" (UniqueName: \"kubernetes.io/projected/d9d5c006-a279-4b85-9394-bb409bccbdfe-kube-api-access-hmcgr\") pod \"d9d5c006-a279-4b85-9394-bb409bccbdfe\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") "
Sep 16 18:17:56.629170 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629172    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-service-ca\") pod \"d9d5c006-a279-4b85-9394-bb409bccbdfe\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") "
Sep 16 18:17:56.629389 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629195    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-oauth-config\") pod \"d9d5c006-a279-4b85-9394-bb409bccbdfe\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") "
Sep 16 18:17:56.629389 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629233    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-trusted-ca-bundle\") pod \"d9d5c006-a279-4b85-9394-bb409bccbdfe\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") "
Sep 16 18:17:56.629489 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629384    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-serving-cert\") pod \"d9d5c006-a279-4b85-9394-bb409bccbdfe\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") "
Sep 16 18:17:56.629550 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629529    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-oauth-serving-cert\") pod \"d9d5c006-a279-4b85-9394-bb409bccbdfe\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") "
Sep 16 18:17:56.629605 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629557    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-config\") pod \"d9d5c006-a279-4b85-9394-bb409bccbdfe\" (UID: \"d9d5c006-a279-4b85-9394-bb409bccbdfe\") "
Sep 16 18:17:56.629658 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629593    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d9d5c006-a279-4b85-9394-bb409bccbdfe" (UID: "d9d5c006-a279-4b85-9394-bb409bccbdfe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:17:56.629775 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629670    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-service-ca" (OuterVolumeSpecName: "service-ca") pod "d9d5c006-a279-4b85-9394-bb409bccbdfe" (UID: "d9d5c006-a279-4b85-9394-bb409bccbdfe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:17:56.629858 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629836    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:17:56.629923 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629856    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-trusted-ca-bundle\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:17:56.629972 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.629859    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d9d5c006-a279-4b85-9394-bb409bccbdfe" (UID: "d9d5c006-a279-4b85-9394-bb409bccbdfe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:17:56.630043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.630020    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-config" (OuterVolumeSpecName: "console-config") pod "d9d5c006-a279-4b85-9394-bb409bccbdfe" (UID: "d9d5c006-a279-4b85-9394-bb409bccbdfe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:17:56.631637 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.631607    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d5c006-a279-4b85-9394-bb409bccbdfe-kube-api-access-hmcgr" (OuterVolumeSpecName: "kube-api-access-hmcgr") pod "d9d5c006-a279-4b85-9394-bb409bccbdfe" (UID: "d9d5c006-a279-4b85-9394-bb409bccbdfe"). InnerVolumeSpecName "kube-api-access-hmcgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:17:56.631776 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.631675    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d9d5c006-a279-4b85-9394-bb409bccbdfe" (UID: "d9d5c006-a279-4b85-9394-bb409bccbdfe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:17:56.631776 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.631724    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d9d5c006-a279-4b85-9394-bb409bccbdfe" (UID: "d9d5c006-a279-4b85-9394-bb409bccbdfe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:17:56.730720 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.730691    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-oauth-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:17:56.730720 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.730716    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:17:56.730869 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.730729    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmcgr\" (UniqueName: \"kubernetes.io/projected/d9d5c006-a279-4b85-9394-bb409bccbdfe-kube-api-access-hmcgr\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:17:56.730869 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.730741    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-oauth-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:17:56.730869 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:56.730767    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9d5c006-a279-4b85-9394-bb409bccbdfe-console-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:17:57.527896 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:57.527865    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59475bf445-9x5hc"
Sep 16 18:17:57.547015 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:57.546987    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59475bf445-9x5hc"]
Sep 16 18:17:57.552065 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:57.552044    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59475bf445-9x5hc"]
Sep 16 18:17:58.360455 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:58.360423    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sdc6h"
Sep 16 18:17:58.947097 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:17:58.947062    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9d5c006-a279-4b85-9394-bb409bccbdfe" path="/var/lib/kubelet/pods/d9d5c006-a279-4b85-9394-bb409bccbdfe/volumes"
Sep 16 18:18:01.464310 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.464251    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78d5b84795-vmg5c" podUID="d6edff61-0ad4-465e-a179-32b283bd6ce4" containerName="console" containerID="cri-o://9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72" gracePeriod=15
Sep 16 18:18:01.729033 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.728976    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:18:01.872957 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.872924    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-config\") pod \"d6edff61-0ad4-465e-a179-32b283bd6ce4\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") "
Sep 16 18:18:01.873131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.872971    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-service-ca\") pod \"d6edff61-0ad4-465e-a179-32b283bd6ce4\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") "
Sep 16 18:18:01.873131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.873001    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-serving-cert\") pod \"d6edff61-0ad4-465e-a179-32b283bd6ce4\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") "
Sep 16 18:18:01.873131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.873039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-oauth-serving-cert\") pod \"d6edff61-0ad4-465e-a179-32b283bd6ce4\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") "
Sep 16 18:18:01.873131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.873093    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-oauth-config\") pod \"d6edff61-0ad4-465e-a179-32b283bd6ce4\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") "
Sep 16 18:18:01.873131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.873115    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-trusted-ca-bundle\") pod \"d6edff61-0ad4-465e-a179-32b283bd6ce4\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") "
Sep 16 18:18:01.873406 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.873148    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ld2vj\" (UniqueName: \"kubernetes.io/projected/d6edff61-0ad4-465e-a179-32b283bd6ce4-kube-api-access-ld2vj\") pod \"d6edff61-0ad4-465e-a179-32b283bd6ce4\" (UID: \"d6edff61-0ad4-465e-a179-32b283bd6ce4\") "
Sep 16 18:18:01.873473 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.873441    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-config" (OuterVolumeSpecName: "console-config") pod "d6edff61-0ad4-465e-a179-32b283bd6ce4" (UID: "d6edff61-0ad4-465e-a179-32b283bd6ce4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:18:01.873473 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.873463    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-service-ca" (OuterVolumeSpecName: "service-ca") pod "d6edff61-0ad4-465e-a179-32b283bd6ce4" (UID: "d6edff61-0ad4-465e-a179-32b283bd6ce4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:18:01.873604 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.873583    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d6edff61-0ad4-465e-a179-32b283bd6ce4" (UID: "d6edff61-0ad4-465e-a179-32b283bd6ce4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:18:01.873730 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.873710    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d6edff61-0ad4-465e-a179-32b283bd6ce4" (UID: "d6edff61-0ad4-465e-a179-32b283bd6ce4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:18:01.875387 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.875355    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d6edff61-0ad4-465e-a179-32b283bd6ce4" (UID: "d6edff61-0ad4-465e-a179-32b283bd6ce4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:18:01.875387 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.875362    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6edff61-0ad4-465e-a179-32b283bd6ce4-kube-api-access-ld2vj" (OuterVolumeSpecName: "kube-api-access-ld2vj") pod "d6edff61-0ad4-465e-a179-32b283bd6ce4" (UID: "d6edff61-0ad4-465e-a179-32b283bd6ce4"). InnerVolumeSpecName "kube-api-access-ld2vj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:18:01.875526 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.875401    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d6edff61-0ad4-465e-a179-32b283bd6ce4" (UID: "d6edff61-0ad4-465e-a179-32b283bd6ce4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:18:01.973984 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.973943    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-oauth-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:01.973984 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.973976    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-oauth-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:01.973984 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.973988    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-trusted-ca-bundle\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:01.974247 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.974000    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld2vj\" (UniqueName: \"kubernetes.io/projected/d6edff61-0ad4-465e-a179-32b283bd6ce4-kube-api-access-ld2vj\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:01.974247 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.974012    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:01.974247 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.974024    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d6edff61-0ad4-465e-a179-32b283bd6ce4-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:01.974247 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:01.974035    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d6edff61-0ad4-465e-a179-32b283bd6ce4-console-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:02.549381 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.549351    2576 generic.go:356] "Generic (PLEG): container finished" podID="d6edff61-0ad4-465e-a179-32b283bd6ce4" containerID="9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72" exitCode=0
Sep 16 18:18:02.549767 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.549393    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d5b84795-vmg5c"
Sep 16 18:18:02.549767 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.549414    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d5b84795-vmg5c" event={"ID":"d6edff61-0ad4-465e-a179-32b283bd6ce4","Type":"ContainerDied","Data":"9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72"}
Sep 16 18:18:02.549767 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.549452    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d5b84795-vmg5c" event={"ID":"d6edff61-0ad4-465e-a179-32b283bd6ce4","Type":"ContainerDied","Data":"b7784706a23b0da8920885024ca762e0c97d8c56a28137f7343613644ac64776"}
Sep 16 18:18:02.549767 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.549472    2576 scope.go:117] "RemoveContainer" containerID="9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72"
Sep 16 18:18:02.558328 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.558313    2576 scope.go:117] "RemoveContainer" containerID="9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72"
Sep 16 18:18:02.558590 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:18:02.558569    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72\": container with ID starting with 9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72 not found: ID does not exist" containerID="9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72"
Sep 16 18:18:02.558641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.558598    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72"} err="failed to get container status \"9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72\": rpc error: code = NotFound desc = could not find container \"9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72\": container with ID starting with 9dcd70f6bb5a8be789e0f4cac64c148ebea48a93d87d74deb9ef3c0150f09a72 not found: ID does not exist"
Sep 16 18:18:02.570809 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.570788    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78d5b84795-vmg5c"]
Sep 16 18:18:02.572977 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.572960    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78d5b84795-vmg5c"]
Sep 16 18:18:02.943054 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:02.943022    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6edff61-0ad4-465e-a179-32b283bd6ce4" path="/var/lib/kubelet/pods/d6edff61-0ad4-465e-a179-32b283bd6ce4/volumes"
Sep 16 18:18:10.568895 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.568672    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b67d87dd-2xbml" podUID="275e3077-1203-4f0f-88cb-3b30d0995b31" containerName="console" containerID="cri-o://ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da" gracePeriod=15
Sep 16 18:18:10.827895 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.827134    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:18:10.833148 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.833069    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:18:10.834133 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.834076    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6876bb786b-5ssjl"
Sep 16 18:18:10.949171 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949139    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-oauth-serving-cert\") pod \"275e3077-1203-4f0f-88cb-3b30d0995b31\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") "
Sep 16 18:18:10.949371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949220    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5s69\" (UniqueName: \"kubernetes.io/projected/275e3077-1203-4f0f-88cb-3b30d0995b31-kube-api-access-n5s69\") pod \"275e3077-1203-4f0f-88cb-3b30d0995b31\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") "
Sep 16 18:18:10.949371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949268    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-serving-cert\") pod \"275e3077-1203-4f0f-88cb-3b30d0995b31\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") "
Sep 16 18:18:10.949371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949298    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-console-config\") pod \"275e3077-1203-4f0f-88cb-3b30d0995b31\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") "
Sep 16 18:18:10.949371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949335    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-service-ca\") pod \"275e3077-1203-4f0f-88cb-3b30d0995b31\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") "
Sep 16 18:18:10.949586 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949371    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-oauth-config\") pod \"275e3077-1203-4f0f-88cb-3b30d0995b31\" (UID: \"275e3077-1203-4f0f-88cb-3b30d0995b31\") "
Sep 16 18:18:10.949586 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949518    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "275e3077-1203-4f0f-88cb-3b30d0995b31" (UID: "275e3077-1203-4f0f-88cb-3b30d0995b31"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:18:10.949695 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949650    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-oauth-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:10.949695 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949650    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-console-config" (OuterVolumeSpecName: "console-config") pod "275e3077-1203-4f0f-88cb-3b30d0995b31" (UID: "275e3077-1203-4f0f-88cb-3b30d0995b31"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:18:10.949695 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.949677    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-service-ca" (OuterVolumeSpecName: "service-ca") pod "275e3077-1203-4f0f-88cb-3b30d0995b31" (UID: "275e3077-1203-4f0f-88cb-3b30d0995b31"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:18:10.955077 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.955038    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "275e3077-1203-4f0f-88cb-3b30d0995b31" (UID: "275e3077-1203-4f0f-88cb-3b30d0995b31"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:18:10.955751 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.955725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/275e3077-1203-4f0f-88cb-3b30d0995b31-kube-api-access-n5s69" (OuterVolumeSpecName: "kube-api-access-n5s69") pod "275e3077-1203-4f0f-88cb-3b30d0995b31" (UID: "275e3077-1203-4f0f-88cb-3b30d0995b31"). InnerVolumeSpecName "kube-api-access-n5s69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:18:10.956429 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:10.956405    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "275e3077-1203-4f0f-88cb-3b30d0995b31" (UID: "275e3077-1203-4f0f-88cb-3b30d0995b31"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:18:11.051733 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.051686    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5s69\" (UniqueName: \"kubernetes.io/projected/275e3077-1203-4f0f-88cb-3b30d0995b31-kube-api-access-n5s69\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:11.051733 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.051733    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:11.051985 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.051746    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-console-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:11.051985 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.051759    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/275e3077-1203-4f0f-88cb-3b30d0995b31-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:11.051985 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.051770    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/275e3077-1203-4f0f-88cb-3b30d0995b31-console-oauth-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:18:11.588189 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.588149    2576 generic.go:356] "Generic (PLEG): container finished" podID="275e3077-1203-4f0f-88cb-3b30d0995b31" containerID="ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da" exitCode=0
Sep 16 18:18:11.588189 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.588194    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b67d87dd-2xbml"
Sep 16 18:18:11.588659 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.588240    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b67d87dd-2xbml" event={"ID":"275e3077-1203-4f0f-88cb-3b30d0995b31","Type":"ContainerDied","Data":"ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da"}
Sep 16 18:18:11.588659 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.588286    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b67d87dd-2xbml" event={"ID":"275e3077-1203-4f0f-88cb-3b30d0995b31","Type":"ContainerDied","Data":"92123028b9fffbc0f287eacec630e37281b02a2b3ba53fa02cb9d17989b7fc52"}
Sep 16 18:18:11.588659 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.588305    2576 scope.go:117] "RemoveContainer" containerID="ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da"
Sep 16 18:18:11.597080 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.597065    2576 scope.go:117] "RemoveContainer" containerID="ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da"
Sep 16 18:18:11.597350 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:18:11.597333    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da\": container with ID starting with ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da not found: ID does not exist" containerID="ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da"
Sep 16 18:18:11.597405 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.597359    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da"} err="failed to get container status \"ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da\": rpc error: code = NotFound desc = could not find container \"ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da\": container with ID starting with ad24c697a9823e44b240653aed6a4f0f464dd2894f61b938b336fc1c1499d4da not found: ID does not exist"
Sep 16 18:18:11.609763 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.609740    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b67d87dd-2xbml"]
Sep 16 18:18:11.615993 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:11.615975    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b67d87dd-2xbml"]
Sep 16 18:18:12.943546 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:18:12.943509    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="275e3077-1203-4f0f-88cb-3b30d0995b31" path="/var/lib/kubelet/pods/275e3077-1203-4f0f-88cb-3b30d0995b31/volumes"
Sep 16 18:21:16.859431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:21:16.859407    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 18:22:31.485085 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.485052    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:22:31.485660 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.485350    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="alertmanager" containerID="cri-o://a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4" gracePeriod=120
Sep 16 18:22:31.485660 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.485428    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy-metric" containerID="cri-o://bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345" gracePeriod=120
Sep 16 18:22:31.485660 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.485478    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy" containerID="cri-o://190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8" gracePeriod=120
Sep 16 18:22:31.485660 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.485428    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy-web" containerID="cri-o://f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc" gracePeriod=120
Sep 16 18:22:31.485660 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.485472    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="config-reloader" containerID="cri-o://af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19" gracePeriod=120
Sep 16 18:22:31.485660 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.485582    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="prom-label-proxy" containerID="cri-o://8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45" gracePeriod=120
Sep 16 18:22:31.720402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.720379    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:31.752273 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752185    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-main-db\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752273 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752232    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-metrics-client-ca\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752273 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752252    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-trusted-ca-bundle\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752273 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752272    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752388    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-volume\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752425    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-tls-assets\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752458    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752516    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z47j\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-kube-api-access-9z47j\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752547    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-out\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752547    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:22:31.752731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752593    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-web\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752600    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:22:31.752731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752608    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:22:31.752731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752632    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-cluster-tls-config\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752658    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-web-config\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.752731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.752723    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-main-tls\") pod \"f04ad3b7-bc0a-45de-b13e-1e754baced18\" (UID: \"f04ad3b7-bc0a-45de-b13e-1e754baced18\") "
Sep 16 18:22:31.753522 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.753191    2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-main-db\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.753522 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.753238    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-metrics-client-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.753522 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.753253    2576 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f04ad3b7-bc0a-45de-b13e-1e754baced18-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.755011 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.754980    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:22:31.755445 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.755417    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:22:31.755611 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.755588    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-out" (OuterVolumeSpecName: "config-out") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:22:31.755822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.755798    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-volume" (OuterVolumeSpecName: "config-volume") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:22:31.756082 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.756048    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:22:31.756156 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.756124    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:22:31.756414 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.756392    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:22:31.756580 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.756562    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-kube-api-access-9z47j" (OuterVolumeSpecName: "kube-api-access-9z47j") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "kube-api-access-9z47j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:22:31.759805 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.759745    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:22:31.767183 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.767162    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-web-config" (OuterVolumeSpecName: "web-config") pod "f04ad3b7-bc0a-45de-b13e-1e754baced18" (UID: "f04ad3b7-bc0a-45de-b13e-1e754baced18"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:22:31.853992 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.853965    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.853992 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.853989    2576 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-cluster-tls-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.854144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.853998    2576 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-web-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.854144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.854007    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-main-tls\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.854144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.854017    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.854144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.854026    2576 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.854144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.854034    2576 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-tls-assets\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.854144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.854041    2576 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f04ad3b7-bc0a-45de-b13e-1e754baced18-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.854144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.854049    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z47j\" (UniqueName: \"kubernetes.io/projected/f04ad3b7-bc0a-45de-b13e-1e754baced18-kube-api-access-9z47j\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:31.854144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:31.854057    2576 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f04ad3b7-bc0a-45de-b13e-1e754baced18-config-out\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:22:32.562569 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562535    2576 generic.go:356] "Generic (PLEG): container finished" podID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerID="8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45" exitCode=0
Sep 16 18:22:32.562569 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562559    2576 generic.go:356] "Generic (PLEG): container finished" podID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerID="bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345" exitCode=0
Sep 16 18:22:32.562569 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562569    2576 generic.go:356] "Generic (PLEG): container finished" podID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerID="190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8" exitCode=0
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562578    2576 generic.go:356] "Generic (PLEG): container finished" podID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerID="f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc" exitCode=0
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562586    2576 generic.go:356] "Generic (PLEG): container finished" podID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerID="af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19" exitCode=0
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562593    2576 generic.go:356] "Generic (PLEG): container finished" podID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerID="a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4" exitCode=0
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562618    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562617    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerDied","Data":"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"}
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562746    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerDied","Data":"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"}
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562776    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerDied","Data":"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"}
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562801    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerDied","Data":"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"}
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562817    2576 scope.go:117] "RemoveContainer" containerID="8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562824    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerDied","Data":"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"}
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562849    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerDied","Data":"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"}
Sep 16 18:22:32.563029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.562874    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f04ad3b7-bc0a-45de-b13e-1e754baced18","Type":"ContainerDied","Data":"99730efeaaacbbd978cbfb4e0a6279940238fb55b24b987a5a4f9e30efc9701d"}
Sep 16 18:22:32.572476 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.572459    2576 scope.go:117] "RemoveContainer" containerID="bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"
Sep 16 18:22:32.579299 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.579282    2576 scope.go:117] "RemoveContainer" containerID="190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"
Sep 16 18:22:32.585672 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.585656    2576 scope.go:117] "RemoveContainer" containerID="f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"
Sep 16 18:22:32.589787 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.589763    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:22:32.592603 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.592587    2576 scope.go:117] "RemoveContainer" containerID="af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"
Sep 16 18:22:32.597039 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.597021    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:22:32.599798 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.599785    2576 scope.go:117] "RemoveContainer" containerID="a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"
Sep 16 18:22:32.606327 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.606306    2576 scope.go:117] "RemoveContainer" containerID="75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"
Sep 16 18:22:32.612720 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.612706    2576 scope.go:117] "RemoveContainer" containerID="8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"
Sep 16 18:22:32.612961 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:22:32.612944    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": container with ID starting with 8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45 not found: ID does not exist" containerID="8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"
Sep 16 18:22:32.613023 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.612966    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"} err="failed to get container status \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": rpc error: code = NotFound desc = could not find container \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": container with ID starting with 8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45 not found: ID does not exist"
Sep 16 18:22:32.613023 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.612982    2576 scope.go:117] "RemoveContainer" containerID="bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"
Sep 16 18:22:32.613238 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:22:32.613203    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": container with ID starting with bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345 not found: ID does not exist" containerID="bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"
Sep 16 18:22:32.613304 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.613243    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"} err="failed to get container status \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": rpc error: code = NotFound desc = could not find container \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": container with ID starting with bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345 not found: ID does not exist"
Sep 16 18:22:32.613304 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.613255    2576 scope.go:117] "RemoveContainer" containerID="190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"
Sep 16 18:22:32.613472 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:22:32.613455    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": container with ID starting with 190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8 not found: ID does not exist" containerID="190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"
Sep 16 18:22:32.613505 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.613480    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"} err="failed to get container status \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": rpc error: code = NotFound desc = could not find container \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": container with ID starting with 190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8 not found: ID does not exist"
Sep 16 18:22:32.613505 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.613498    2576 scope.go:117] "RemoveContainer" containerID="f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"
Sep 16 18:22:32.613703 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:22:32.613686    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": container with ID starting with f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc not found: ID does not exist" containerID="f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"
Sep 16 18:22:32.613744 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.613707    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"} err="failed to get container status \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": rpc error: code = NotFound desc = could not find container \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": container with ID starting with f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc not found: ID does not exist"
Sep 16 18:22:32.613744 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.613720    2576 scope.go:117] "RemoveContainer" containerID="af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"
Sep 16 18:22:32.613953 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:22:32.613938    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": container with ID starting with af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19 not found: ID does not exist" containerID="af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"
Sep 16 18:22:32.613998 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.613958    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"} err="failed to get container status \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": rpc error: code = NotFound desc = could not find container \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": container with ID starting with af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19 not found: ID does not exist"
Sep 16 18:22:32.613998 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.613970    2576 scope.go:117] "RemoveContainer" containerID="a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"
Sep 16 18:22:32.614183 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:22:32.614169    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": container with ID starting with a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4 not found: ID does not exist" containerID="a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"
Sep 16 18:22:32.614237 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.614185    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"} err="failed to get container status \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": rpc error: code = NotFound desc = could not find container \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": container with ID starting with a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4 not found: ID does not exist"
Sep 16 18:22:32.614237 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.614197    2576 scope.go:117] "RemoveContainer" containerID="75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"
Sep 16 18:22:32.614568 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:22:32.614553    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": container with ID starting with 75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7 not found: ID does not exist" containerID="75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"
Sep 16 18:22:32.614611 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.614570    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"} err="failed to get container status \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": rpc error: code = NotFound desc = could not find container \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": container with ID starting with 75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7 not found: ID does not exist"
Sep 16 18:22:32.614611 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.614582    2576 scope.go:117] "RemoveContainer" containerID="8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"
Sep 16 18:22:32.614798 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.614781    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"} err="failed to get container status \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": rpc error: code = NotFound desc = could not find container \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": container with ID starting with 8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45 not found: ID does not exist"
Sep 16 18:22:32.614835 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.614798    2576 scope.go:117] "RemoveContainer" containerID="bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"
Sep 16 18:22:32.615027 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.615009    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"} err="failed to get container status \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": rpc error: code = NotFound desc = could not find container \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": container with ID starting with bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345 not found: ID does not exist"
Sep 16 18:22:32.615112 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.615029    2576 scope.go:117] "RemoveContainer" containerID="190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"
Sep 16 18:22:32.615377 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.615359    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"} err="failed to get container status \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": rpc error: code = NotFound desc = could not find container \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": container with ID starting with 190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8 not found: ID does not exist"
Sep 16 18:22:32.615422 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.615378    2576 scope.go:117] "RemoveContainer" containerID="f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"
Sep 16 18:22:32.615595 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.615578    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"} err="failed to get container status \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": rpc error: code = NotFound desc = could not find container \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": container with ID starting with f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc not found: ID does not exist"
Sep 16 18:22:32.615639 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.615596    2576 scope.go:117] "RemoveContainer" containerID="af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"
Sep 16 18:22:32.615796 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.615780    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"} err="failed to get container status \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": rpc error: code = NotFound desc = could not find container \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": container with ID starting with af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19 not found: ID does not exist"
Sep 16 18:22:32.615862 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.615799    2576 scope.go:117] "RemoveContainer" containerID="a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"
Sep 16 18:22:32.616031 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616015    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"} err="failed to get container status \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": rpc error: code = NotFound desc = could not find container \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": container with ID starting with a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4 not found: ID does not exist"
Sep 16 18:22:32.616076 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616032    2576 scope.go:117] "RemoveContainer" containerID="75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"
Sep 16 18:22:32.616271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616254    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"} err="failed to get container status \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": rpc error: code = NotFound desc = could not find container \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": container with ID starting with 75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7 not found: ID does not exist"
Sep 16 18:22:32.616340 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616272    2576 scope.go:117] "RemoveContainer" containerID="8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"
Sep 16 18:22:32.616481 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616464    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"} err="failed to get container status \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": rpc error: code = NotFound desc = could not find container \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": container with ID starting with 8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45 not found: ID does not exist"
Sep 16 18:22:32.616529 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616481    2576 scope.go:117] "RemoveContainer" containerID="bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"
Sep 16 18:22:32.616684 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616669    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"} err="failed to get container status \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": rpc error: code = NotFound desc = could not find container \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": container with ID starting with bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345 not found: ID does not exist"
Sep 16 18:22:32.616684 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616683    2576 scope.go:117] "RemoveContainer" containerID="190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"
Sep 16 18:22:32.616888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616873    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"} err="failed to get container status \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": rpc error: code = NotFound desc = could not find container \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": container with ID starting with 190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8 not found: ID does not exist"
Sep 16 18:22:32.616942 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.616888    2576 scope.go:117] "RemoveContainer" containerID="f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"
Sep 16 18:22:32.617101 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617084    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"} err="failed to get container status \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": rpc error: code = NotFound desc = could not find container \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": container with ID starting with f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc not found: ID does not exist"
Sep 16 18:22:32.617148 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617101    2576 scope.go:117] "RemoveContainer" containerID="af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"
Sep 16 18:22:32.617385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617367    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"} err="failed to get container status \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": rpc error: code = NotFound desc = could not find container \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": container with ID starting with af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19 not found: ID does not exist"
Sep 16 18:22:32.617445 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617385    2576 scope.go:117] "RemoveContainer" containerID="a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"
Sep 16 18:22:32.617607 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617587    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"} err="failed to get container status \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": rpc error: code = NotFound desc = could not find container \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": container with ID starting with a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4 not found: ID does not exist"
Sep 16 18:22:32.617674 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617608    2576 scope.go:117] "RemoveContainer" containerID="75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"
Sep 16 18:22:32.617819 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617804    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"} err="failed to get container status \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": rpc error: code = NotFound desc = could not find container \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": container with ID starting with 75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7 not found: ID does not exist"
Sep 16 18:22:32.617861 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617819    2576 scope.go:117] "RemoveContainer" containerID="8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"
Sep 16 18:22:32.617999 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617983    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"} err="failed to get container status \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": rpc error: code = NotFound desc = could not find container \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": container with ID starting with 8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45 not found: ID does not exist"
Sep 16 18:22:32.618042 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.617999    2576 scope.go:117] "RemoveContainer" containerID="bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"
Sep 16 18:22:32.618188 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.618172    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"} err="failed to get container status \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": rpc error: code = NotFound desc = could not find container \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": container with ID starting with bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345 not found: ID does not exist"
Sep 16 18:22:32.618249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.618188    2576 scope.go:117] "RemoveContainer" containerID="190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"
Sep 16 18:22:32.618428 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.618411    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"} err="failed to get container status \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": rpc error: code = NotFound desc = could not find container \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": container with ID starting with 190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8 not found: ID does not exist"
Sep 16 18:22:32.618495 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.618429    2576 scope.go:117] "RemoveContainer" containerID="f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"
Sep 16 18:22:32.618632 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.618616    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"} err="failed to get container status \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": rpc error: code = NotFound desc = could not find container \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": container with ID starting with f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc not found: ID does not exist"
Sep 16 18:22:32.618676 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.618632    2576 scope.go:117] "RemoveContainer" containerID="af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"
Sep 16 18:22:32.618816 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.618799    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"} err="failed to get container status \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": rpc error: code = NotFound desc = could not find container \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": container with ID starting with af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19 not found: ID does not exist"
Sep 16 18:22:32.618886 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.618816    2576 scope.go:117] "RemoveContainer" containerID="a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"
Sep 16 18:22:32.619031 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619015    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"} err="failed to get container status \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": rpc error: code = NotFound desc = could not find container \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": container with ID starting with a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4 not found: ID does not exist"
Sep 16 18:22:32.619088 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619032    2576 scope.go:117] "RemoveContainer" containerID="75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"
Sep 16 18:22:32.619243 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619228    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"} err="failed to get container status \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": rpc error: code = NotFound desc = could not find container \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": container with ID starting with 75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7 not found: ID does not exist"
Sep 16 18:22:32.619293 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619243    2576 scope.go:117] "RemoveContainer" containerID="8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"
Sep 16 18:22:32.619421 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619406    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"} err="failed to get container status \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": rpc error: code = NotFound desc = could not find container \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": container with ID starting with 8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45 not found: ID does not exist"
Sep 16 18:22:32.619465 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619422    2576 scope.go:117] "RemoveContainer" containerID="bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"
Sep 16 18:22:32.619598 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619581    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"} err="failed to get container status \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": rpc error: code = NotFound desc = could not find container \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": container with ID starting with bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345 not found: ID does not exist"
Sep 16 18:22:32.619663 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619599    2576 scope.go:117] "RemoveContainer" containerID="190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"
Sep 16 18:22:32.619789 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619774    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"} err="failed to get container status \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": rpc error: code = NotFound desc = could not find container \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": container with ID starting with 190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8 not found: ID does not exist"
Sep 16 18:22:32.619789 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619788    2576 scope.go:117] "RemoveContainer" containerID="f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"
Sep 16 18:22:32.619957 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619940    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"} err="failed to get container status \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": rpc error: code = NotFound desc = could not find container \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": container with ID starting with f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc not found: ID does not exist"
Sep 16 18:22:32.619957 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.619956    2576 scope.go:117] "RemoveContainer" containerID="af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"
Sep 16 18:22:32.620144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620130    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"} err="failed to get container status \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": rpc error: code = NotFound desc = could not find container \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": container with ID starting with af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19 not found: ID does not exist"
Sep 16 18:22:32.620184 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620144    2576 scope.go:117] "RemoveContainer" containerID="a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"
Sep 16 18:22:32.620388 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620373    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"} err="failed to get container status \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": rpc error: code = NotFound desc = could not find container \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": container with ID starting with a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4 not found: ID does not exist"
Sep 16 18:22:32.620388 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620387    2576 scope.go:117] "RemoveContainer" containerID="75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"
Sep 16 18:22:32.620562 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620545    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"} err="failed to get container status \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": rpc error: code = NotFound desc = could not find container \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": container with ID starting with 75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7 not found: ID does not exist"
Sep 16 18:22:32.620562 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620561    2576 scope.go:117] "RemoveContainer" containerID="8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"
Sep 16 18:22:32.620766 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620751    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45"} err="failed to get container status \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": rpc error: code = NotFound desc = could not find container \"8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45\": container with ID starting with 8b83f63101943827028677d1b70bc2ec6ad773c0dfdebc5a7b623fb94f2a2a45 not found: ID does not exist"
Sep 16 18:22:32.620766 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620764    2576 scope.go:117] "RemoveContainer" containerID="bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"
Sep 16 18:22:32.620942 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620928    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345"} err="failed to get container status \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": rpc error: code = NotFound desc = could not find container \"bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345\": container with ID starting with bafc00545f418f1fd4e0691adb6714073bbb0fd6e60ba18cdeab235180e88345 not found: ID does not exist"
Sep 16 18:22:32.621015 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.620942    2576 scope.go:117] "RemoveContainer" containerID="190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"
Sep 16 18:22:32.621146 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.621132    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8"} err="failed to get container status \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": rpc error: code = NotFound desc = could not find container \"190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8\": container with ID starting with 190f500a291bacf03ee733239c04a72c462450cafb87197be085e81fea3be3f8 not found: ID does not exist"
Sep 16 18:22:32.621189 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.621146    2576 scope.go:117] "RemoveContainer" containerID="f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"
Sep 16 18:22:32.621373 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.621356    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc"} err="failed to get container status \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": rpc error: code = NotFound desc = could not find container \"f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc\": container with ID starting with f678dd2e9c3d52bde11aef73ec2ed06af5708ceace90ed9d6618895968019abc not found: ID does not exist"
Sep 16 18:22:32.621373 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.621372    2576 scope.go:117] "RemoveContainer" containerID="af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"
Sep 16 18:22:32.621567 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.621551    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19"} err="failed to get container status \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": rpc error: code = NotFound desc = could not find container \"af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19\": container with ID starting with af13281115956bd83619ce42276066416640aa812a4d6e296359fea712c99c19 not found: ID does not exist"
Sep 16 18:22:32.621612 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.621567    2576 scope.go:117] "RemoveContainer" containerID="a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"
Sep 16 18:22:32.621738 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.621722    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4"} err="failed to get container status \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": rpc error: code = NotFound desc = could not find container \"a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4\": container with ID starting with a6878ef230915774a3397edfc763c08a2e27953cb6858a65897a64971553faf4 not found: ID does not exist"
Sep 16 18:22:32.621778 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.621738    2576 scope.go:117] "RemoveContainer" containerID="75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"
Sep 16 18:22:32.621940 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.621922    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7"} err="failed to get container status \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": rpc error: code = NotFound desc = could not find container \"75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7\": container with ID starting with 75c87b230106a4d5718ba65a4590514434d195e3909dc3800634cee21cd721b7 not found: ID does not exist"
Sep 16 18:22:32.636989 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.636966    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:22:32.637267 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637254    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="275e3077-1203-4f0f-88cb-3b30d0995b31" containerName="console"
Sep 16 18:22:32.637308 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637269    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="275e3077-1203-4f0f-88cb-3b30d0995b31" containerName="console"
Sep 16 18:22:32.637308 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637281    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6edff61-0ad4-465e-a179-32b283bd6ce4" containerName="console"
Sep 16 18:22:32.637308 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637287    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6edff61-0ad4-465e-a179-32b283bd6ce4" containerName="console"
Sep 16 18:22:32.637308 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637294    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="alertmanager"
Sep 16 18:22:32.637308 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637299    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="alertmanager"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637313    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy-web"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637319    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy-web"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637325    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy-metric"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637330    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy-metric"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637336    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="prom-label-proxy"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637341    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="prom-label-proxy"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637350    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="config-reloader"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637355    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="config-reloader"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637361    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9d5c006-a279-4b85-9394-bb409bccbdfe" containerName="console"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637366    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9d5c006-a279-4b85-9394-bb409bccbdfe" containerName="console"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637372    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="init-config-reloader"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637377    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="init-config-reloader"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637383    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637387    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637425    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9d5c006-a279-4b85-9394-bb409bccbdfe" containerName="console"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637433    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy-web"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637439    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637444    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6edff61-0ad4-465e-a179-32b283bd6ce4" containerName="console"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637450    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="kube-rbac-proxy-metric"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637456    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="config-reloader"
Sep 16 18:22:32.637453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637462    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="prom-label-proxy"
Sep 16 18:22:32.638064 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637467    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="275e3077-1203-4f0f-88cb-3b30d0995b31" containerName="console"
Sep 16 18:22:32.638064 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.637472    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" containerName="alertmanager"
Sep 16 18:22:32.640971 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.640956    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.644575 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.644558    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 18:22:32.644675 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.644558    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 18:22:32.645651 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.645633    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 18:22:32.645729 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.645713    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 18:22:32.645801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.645786    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 18:22:32.646584 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.646563    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c24k4\""
Sep 16 18:22:32.646682 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.646594    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 18:22:32.646682 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.646597    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 18:22:32.646872 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.646854    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 18:22:32.660027 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.660010    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 18:22:32.674833 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.674809    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:22:32.758941 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.758878    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.758941 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.758905    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.758941 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.758930    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759115 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.758977    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759115 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.759017    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1df6ea65-4988-497f-9e20-a65d19771b12-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759115 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.759079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.759110    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1df6ea65-4988-497f-9e20-a65d19771b12-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.759140    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1df6ea65-4988-497f-9e20-a65d19771b12-config-out\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.759165    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1df6ea65-4988-497f-9e20-a65d19771b12-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.759185    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1df6ea65-4988-497f-9e20-a65d19771b12-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.759202    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-web-config\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.759238    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48dxf\" (UniqueName: \"kubernetes.io/projected/1df6ea65-4988-497f-9e20-a65d19771b12-kube-api-access-48dxf\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.759477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.759274    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-config-volume\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.859943 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.859920    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1df6ea65-4988-497f-9e20-a65d19771b12-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860050 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.859953    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860050 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.859971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1df6ea65-4988-497f-9e20-a65d19771b12-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860050 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.859986    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1df6ea65-4988-497f-9e20-a65d19771b12-config-out\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860050 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860004    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1df6ea65-4988-497f-9e20-a65d19771b12-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860267 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1df6ea65-4988-497f-9e20-a65d19771b12-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860267 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-web-config\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860267 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48dxf\" (UniqueName: \"kubernetes.io/projected/1df6ea65-4988-497f-9e20-a65d19771b12-kube-api-access-48dxf\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860267 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860241    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-config-volume\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860374    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860689 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1df6ea65-4988-497f-9e20-a65d19771b12-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860787 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860761    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1df6ea65-4988-497f-9e20-a65d19771b12-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.860989 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.860963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1df6ea65-4988-497f-9e20-a65d19771b12-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.863095 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.863073    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1df6ea65-4988-497f-9e20-a65d19771b12-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.863202 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.863150    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1df6ea65-4988-497f-9e20-a65d19771b12-config-out\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.863571 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.863343    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-config-volume\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.863571 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.863504    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.863706 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.863619    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.863706 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.863691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.863874 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.863852    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.864109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.864091    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.864845 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.864829    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1df6ea65-4988-497f-9e20-a65d19771b12-web-config\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.874125 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.874106    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48dxf\" (UniqueName: \"kubernetes.io/projected/1df6ea65-4988-497f-9e20-a65d19771b12-kube-api-access-48dxf\") pod \"alertmanager-main-0\" (UID: \"1df6ea65-4988-497f-9e20-a65d19771b12\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:32.943738 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.943716    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f04ad3b7-bc0a-45de-b13e-1e754baced18" path="/var/lib/kubelet/pods/f04ad3b7-bc0a-45de-b13e-1e754baced18/volumes"
Sep 16 18:22:32.949859 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:32.949843    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 18:22:33.233347 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:33.233320    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 18:22:33.235798 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:22:33.235774    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df6ea65_4988_497f_9e20_a65d19771b12.slice/crio-546bc5fb31371e4590657c1d269e5f8bd83d97718f6d1205cae8833bb94414fa WatchSource:0}: Error finding container 546bc5fb31371e4590657c1d269e5f8bd83d97718f6d1205cae8833bb94414fa: Status 404 returned error can't find the container with id 546bc5fb31371e4590657c1d269e5f8bd83d97718f6d1205cae8833bb94414fa
Sep 16 18:22:33.568962 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:33.568883    2576 generic.go:356] "Generic (PLEG): container finished" podID="1df6ea65-4988-497f-9e20-a65d19771b12" containerID="a108a627e4b7b8178879fbed0fbc221098030ad653530ce10ab24c261ab31667" exitCode=0
Sep 16 18:22:33.569349 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:33.568978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1df6ea65-4988-497f-9e20-a65d19771b12","Type":"ContainerDied","Data":"a108a627e4b7b8178879fbed0fbc221098030ad653530ce10ab24c261ab31667"}
Sep 16 18:22:33.569349 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:33.569059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1df6ea65-4988-497f-9e20-a65d19771b12","Type":"ContainerStarted","Data":"546bc5fb31371e4590657c1d269e5f8bd83d97718f6d1205cae8833bb94414fa"}
Sep 16 18:22:34.576188 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:34.576155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1df6ea65-4988-497f-9e20-a65d19771b12","Type":"ContainerStarted","Data":"df701710a888c1823cec17fa022a65953ccf309185bae0c55b892e7f43c012ca"}
Sep 16 18:22:34.576553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:34.576242    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1df6ea65-4988-497f-9e20-a65d19771b12","Type":"ContainerStarted","Data":"f34955a269b53f35ded8917c54501a2de21a6b6818d80b3a1ebd66e790d6c623"}
Sep 16 18:22:34.576553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:34.576272    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1df6ea65-4988-497f-9e20-a65d19771b12","Type":"ContainerStarted","Data":"b125c3293d47bf9ae234faa502e0287ebff2b1eb2d2c23cee9f1710bcd9de68d"}
Sep 16 18:22:34.576553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:34.576297    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1df6ea65-4988-497f-9e20-a65d19771b12","Type":"ContainerStarted","Data":"9486ff2c210980191d7f6590288801c17b4ac41754b53004168770dc696a4107"}
Sep 16 18:22:34.576553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:34.576326    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1df6ea65-4988-497f-9e20-a65d19771b12","Type":"ContainerStarted","Data":"8a546da349b22e2de689770d247e04bb6c3879c2765318b16f08bf9d0b2ac81e"}
Sep 16 18:22:34.576553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:34.576349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1df6ea65-4988-497f-9e20-a65d19771b12","Type":"ContainerStarted","Data":"d0dd1467bb2da58110242480f4fb3377d8c2ac4910fcec62f3598702d0e0ac61"}
Sep 16 18:22:34.636605 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:34.636556    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.6218408650000002 podStartE2EDuration="2.636537254s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.696389ms" isStatefulPod=false podCreationTimestamp="2026-09-16 18:22:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:22:34.635201046 +0000 UTC m=+378.191494395" watchObservedRunningTime="2026-09-16 18:22:34.636537254 +0000 UTC m=+378.192830568"
Sep 16 18:22:36.835660 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.835622    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:22:36.839007 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.838981    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.843189 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.843159    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 18:22:36.843488 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.843460    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 18:22:36.843605 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.843492    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 18:22:36.844446 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.844424    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 18:22:36.844556 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.844530    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 18:22:36.844556 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.844547    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 18:22:36.844672 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.844650    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-aq1eqgg2joohi\""
Sep 16 18:22:36.844720 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.844676    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 18:22:36.844813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.844790    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 18:22:36.844902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.844893    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 18:22:36.844983 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.844968    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 18:22:36.845140 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.845123    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 18:22:36.845288 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.845273    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-28ws8\""
Sep 16 18:22:36.845407 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.845329    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 18:22:36.850026 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.850006    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 18:22:36.850900 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.850881    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 18:22:36.858810 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.858784    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:22:36.892931 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.892901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893134 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66ff755f-26dd-4520-9f52-da84d1ab1e0a-config-out\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893269    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-web-config\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893368    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893548 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893548 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893444    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsn49\" (UniqueName: \"kubernetes.io/projected/66ff755f-26dd-4520-9f52-da84d1ab1e0a-kube-api-access-lsn49\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893548 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893548 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893687 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893564    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893687 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893586    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893687 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893610    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893687 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893654    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66ff755f-26dd-4520-9f52-da84d1ab1e0a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893687 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-config\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893877 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893716    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893877 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893961 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.893961 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.893938    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994546 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994507    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994692 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994692 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994692 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994669    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994692 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsn49\" (UniqueName: \"kubernetes.io/projected/66ff755f-26dd-4520-9f52-da84d1ab1e0a-kube-api-access-lsn49\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994761    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994806    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994822    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.994902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994882    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66ff755f-26dd-4520-9f52-da84d1ab1e0a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-config\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994929    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.994996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.995024    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.995078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.995108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66ff755f-26dd-4520-9f52-da84d1ab1e0a-config-out\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.995134    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.995163    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-web-config\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.995635 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.995462    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.997246 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.996154    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.997444 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.997421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.997669 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.997643    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:36.998309 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:36.998283    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.000289 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.000263    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-config\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.000438 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.000417    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.000542 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.000513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.000890 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.000867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-web-config\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.000981 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.000894    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.000981 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.000916    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.001115 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.001090    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.001174 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.000901    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.001332 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.001298    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66ff755f-26dd-4520-9f52-da84d1ab1e0a-config-out\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.001447 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.001354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.001447 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.001401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.002158 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.002131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66ff755f-26dd-4520-9f52-da84d1ab1e0a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.002981 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.002959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66ff755f-26dd-4520-9f52-da84d1ab1e0a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.003831 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.003813    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsn49\" (UniqueName: \"kubernetes.io/projected/66ff755f-26dd-4520-9f52-da84d1ab1e0a-kube-api-access-lsn49\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.004198 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.004180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66ff755f-26dd-4520-9f52-da84d1ab1e0a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"66ff755f-26dd-4520-9f52-da84d1ab1e0a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.151075 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.151030    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:22:37.447170 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.447141    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 18:22:37.447307 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:22:37.447139    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66ff755f_26dd_4520_9f52_da84d1ab1e0a.slice/crio-e583d81c300e910aa806f412a5ea8d516c9746b84c59c52ee170135a0d92241a WatchSource:0}: Error finding container e583d81c300e910aa806f412a5ea8d516c9746b84c59c52ee170135a0d92241a: Status 404 returned error can't find the container with id e583d81c300e910aa806f412a5ea8d516c9746b84c59c52ee170135a0d92241a
Sep 16 18:22:37.591217 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.591177    2576 generic.go:356] "Generic (PLEG): container finished" podID="66ff755f-26dd-4520-9f52-da84d1ab1e0a" containerID="22144b6edde0f5dbfc010cf9416d0be72854a4c1f22857921124e9d9a8b04c29" exitCode=0
Sep 16 18:22:37.591327 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.591273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66ff755f-26dd-4520-9f52-da84d1ab1e0a","Type":"ContainerDied","Data":"22144b6edde0f5dbfc010cf9416d0be72854a4c1f22857921124e9d9a8b04c29"}
Sep 16 18:22:37.591410 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.591350    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66ff755f-26dd-4520-9f52-da84d1ab1e0a","Type":"ContainerStarted","Data":"e583d81c300e910aa806f412a5ea8d516c9746b84c59c52ee170135a0d92241a"}
Sep 16 18:22:37.597586 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:37.597567    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:22:41.613094 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:41.613063    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66ff755f-26dd-4520-9f52-da84d1ab1e0a","Type":"ContainerStarted","Data":"c6499c135a233a56950a4b73e7f442f13efda4d65217c5e6513fcc4ad4909898"}
Sep 16 18:22:41.613458 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:41.613125    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66ff755f-26dd-4520-9f52-da84d1ab1e0a","Type":"ContainerStarted","Data":"4b973cd2d477fe915804c081157789f7369bb087b44c4c2b3b91d2d9160b489b"}
Sep 16 18:22:41.613458 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:41.613151    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66ff755f-26dd-4520-9f52-da84d1ab1e0a","Type":"ContainerStarted","Data":"f3659e3a09643252bacf4b2a98d6e5b458ff781dc2d3d0ab51da860a1ed33445"}
Sep 16 18:22:41.613458 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:41.613175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66ff755f-26dd-4520-9f52-da84d1ab1e0a","Type":"ContainerStarted","Data":"91013dee352a220ce77c6ab5db9338140b917af511ca5e89c3e5ffa1537f825d"}
Sep 16 18:22:41.613458 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:41.613199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66ff755f-26dd-4520-9f52-da84d1ab1e0a","Type":"ContainerStarted","Data":"14f40a724820149f9ce61a30b84fcd74f5fcd1a3ae25e7de178d31e3b36cc557"}
Sep 16 18:22:41.613458 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:41.613243    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66ff755f-26dd-4520-9f52-da84d1ab1e0a","Type":"ContainerStarted","Data":"f71f7a41c57444ba13289ff6452a2a5f590651904c56bd803492c3e01ddd8d27"}
Sep 16 18:22:42.151418 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:22:42.151378    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:23:37.151435 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:37.151394    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:23:37.170731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:37.170707    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:23:37.206233 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:37.205907    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=57.897915677 podStartE2EDuration="1m1.205885671s" totalImagesPullingTime="3.28652076s" totalInitContainerRuntime="21.449234ms" isStatefulPod=false podCreationTimestamp="2026-09-16 18:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:22:41.670003277 +0000 UTC m=+385.226296621" watchObservedRunningTime="2026-09-16 18:23:37.205885671 +0000 UTC m=+440.762178984"
Sep 16 18:23:37.858739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:37.858715    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 18:23:57.454633 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.454557    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cff88886-5rmd8"]
Sep 16 18:23:57.458245 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.458220    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.473528 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.473507    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cff88886-5rmd8"]
Sep 16 18:23:57.510571 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.510540    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-serving-cert\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.510571 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.510568    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-trusted-ca-bundle\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.510756 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.510586    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-oauth-serving-cert\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.510756 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.510619    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-config\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.510756 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.510653    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zxh9\" (UniqueName: \"kubernetes.io/projected/c3039d09-6266-4ebf-a1ad-7b10780bd812-kube-api-access-6zxh9\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.510756 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.510672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-service-ca\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.510756 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.510712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-oauth-config\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.611456 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.611424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-serving-cert\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.611456 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.611455    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-trusted-ca-bundle\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.611682 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.611471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-oauth-serving-cert\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.611682 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.611532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-config\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.611682 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.611604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zxh9\" (UniqueName: \"kubernetes.io/projected/c3039d09-6266-4ebf-a1ad-7b10780bd812-kube-api-access-6zxh9\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.611682 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.611638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-service-ca\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.611873 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.611699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-oauth-config\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.612369 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.612337    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-oauth-serving-cert\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.612490 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.612401    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-config\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.612490 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.612421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-service-ca\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.612490 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.612471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-trusted-ca-bundle\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.614198 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.614179    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-oauth-config\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.614309 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.614267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-serving-cert\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.620936 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.620915    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zxh9\" (UniqueName: \"kubernetes.io/projected/c3039d09-6266-4ebf-a1ad-7b10780bd812-kube-api-access-6zxh9\") pod \"console-7cff88886-5rmd8\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") " pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:57.767672 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:57.767587    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:23:58.047194 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:58.047109    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cff88886-5rmd8"]
Sep 16 18:23:58.050355 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:23:58.050329    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3039d09_6266_4ebf_a1ad_7b10780bd812.slice/crio-dd652a7f0c68534054f2acd1c71b6c4552a05df31beaa0dadb0e9444183b2b3b WatchSource:0}: Error finding container dd652a7f0c68534054f2acd1c71b6c4552a05df31beaa0dadb0e9444183b2b3b: Status 404 returned error can't find the container with id dd652a7f0c68534054f2acd1c71b6c4552a05df31beaa0dadb0e9444183b2b3b
Sep 16 18:23:58.928696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:58.928659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cff88886-5rmd8" event={"ID":"c3039d09-6266-4ebf-a1ad-7b10780bd812","Type":"ContainerStarted","Data":"f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec"}
Sep 16 18:23:58.929078 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:58.928706    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cff88886-5rmd8" event={"ID":"c3039d09-6266-4ebf-a1ad-7b10780bd812","Type":"ContainerStarted","Data":"dd652a7f0c68534054f2acd1c71b6c4552a05df31beaa0dadb0e9444183b2b3b"}
Sep 16 18:23:58.955878 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:23:58.955828    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cff88886-5rmd8" podStartSLOduration=1.955810968 podStartE2EDuration="1.955810968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:23:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:23:58.955085003 +0000 UTC m=+462.511378313" watchObservedRunningTime="2026-09-16 18:23:58.955810968 +0000 UTC m=+462.512104279"
Sep 16 18:24:07.768016 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:07.767981    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:24:07.768439 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:07.768069    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:24:07.773014 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:07.772984    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:24:07.971072 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:07.971043    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:24:08.017561 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:08.017523    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59dcd55c64-nt9zc"]
Sep 16 18:24:16.890130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:16.890101    2576 scope.go:117] "RemoveContainer" containerID="d9a4994433a9776abe266af8c289059a77b25df3ec5de466a5359822c987f276"
Sep 16 18:24:33.037356 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.037293    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59dcd55c64-nt9zc" podUID="338c680b-fe77-44e7-8c7a-868682f99f49" containerName="console" containerID="cri-o://5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87" gracePeriod=15
Sep 16 18:24:33.276574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.276550    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:24:33.317461 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.317367    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-trusted-ca-bundle\") pod \"338c680b-fe77-44e7-8c7a-868682f99f49\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") "
Sep 16 18:24:33.317641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.317495    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnbct\" (UniqueName: \"kubernetes.io/projected/338c680b-fe77-44e7-8c7a-868682f99f49-kube-api-access-vnbct\") pod \"338c680b-fe77-44e7-8c7a-868682f99f49\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") "
Sep 16 18:24:33.317641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.317527    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-serving-cert\") pod \"338c680b-fe77-44e7-8c7a-868682f99f49\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") "
Sep 16 18:24:33.317641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.317603    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-oauth-serving-cert\") pod \"338c680b-fe77-44e7-8c7a-868682f99f49\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") "
Sep 16 18:24:33.317797 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.317643    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-service-ca\") pod \"338c680b-fe77-44e7-8c7a-868682f99f49\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") "
Sep 16 18:24:33.317797 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.317689    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-console-config\") pod \"338c680b-fe77-44e7-8c7a-868682f99f49\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") "
Sep 16 18:24:33.317797 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.317725    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-oauth-config\") pod \"338c680b-fe77-44e7-8c7a-868682f99f49\" (UID: \"338c680b-fe77-44e7-8c7a-868682f99f49\") "
Sep 16 18:24:33.317797 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.317764    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "338c680b-fe77-44e7-8c7a-868682f99f49" (UID: "338c680b-fe77-44e7-8c7a-868682f99f49"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:24:33.317980 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.317921    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "338c680b-fe77-44e7-8c7a-868682f99f49" (UID: "338c680b-fe77-44e7-8c7a-868682f99f49"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:24:33.318038 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.318016    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-trusted-ca-bundle\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:24:33.318089 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.318039    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-oauth-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:24:33.318089 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.318031    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-console-config" (OuterVolumeSpecName: "console-config") pod "338c680b-fe77-44e7-8c7a-868682f99f49" (UID: "338c680b-fe77-44e7-8c7a-868682f99f49"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:24:33.318089 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.318057    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-service-ca" (OuterVolumeSpecName: "service-ca") pod "338c680b-fe77-44e7-8c7a-868682f99f49" (UID: "338c680b-fe77-44e7-8c7a-868682f99f49"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:24:33.319728 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.319703    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/338c680b-fe77-44e7-8c7a-868682f99f49-kube-api-access-vnbct" (OuterVolumeSpecName: "kube-api-access-vnbct") pod "338c680b-fe77-44e7-8c7a-868682f99f49" (UID: "338c680b-fe77-44e7-8c7a-868682f99f49"). InnerVolumeSpecName "kube-api-access-vnbct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:24:33.319728 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.319709    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "338c680b-fe77-44e7-8c7a-868682f99f49" (UID: "338c680b-fe77-44e7-8c7a-868682f99f49"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:24:33.319893 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.319870    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "338c680b-fe77-44e7-8c7a-868682f99f49" (UID: "338c680b-fe77-44e7-8c7a-868682f99f49"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:24:33.418739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.418704    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:24:33.418739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.418732    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/338c680b-fe77-44e7-8c7a-868682f99f49-console-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:24:33.418739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.418742    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-oauth-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:24:33.418739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.418751    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnbct\" (UniqueName: \"kubernetes.io/projected/338c680b-fe77-44e7-8c7a-868682f99f49-kube-api-access-vnbct\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:24:33.419003 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:33.418760    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/338c680b-fe77-44e7-8c7a-868682f99f49-console-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:24:34.073222 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.073183    2576 generic.go:356] "Generic (PLEG): container finished" podID="338c680b-fe77-44e7-8c7a-868682f99f49" containerID="5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87" exitCode=0
Sep 16 18:24:34.073655 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.073236    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59dcd55c64-nt9zc"
Sep 16 18:24:34.073655 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.073261    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dcd55c64-nt9zc" event={"ID":"338c680b-fe77-44e7-8c7a-868682f99f49","Type":"ContainerDied","Data":"5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87"}
Sep 16 18:24:34.073655 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.073314    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59dcd55c64-nt9zc" event={"ID":"338c680b-fe77-44e7-8c7a-868682f99f49","Type":"ContainerDied","Data":"1d55beb66c00d0e2b4f76edbb01f666c17a85156b227fb81b37911ed6bf37f57"}
Sep 16 18:24:34.073655 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.073334    2576 scope.go:117] "RemoveContainer" containerID="5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87"
Sep 16 18:24:34.082470 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.082451    2576 scope.go:117] "RemoveContainer" containerID="5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87"
Sep 16 18:24:34.082781 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:24:34.082763    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87\": container with ID starting with 5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87 not found: ID does not exist" containerID="5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87"
Sep 16 18:24:34.082828 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.082792    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87"} err="failed to get container status \"5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87\": rpc error: code = NotFound desc = could not find container \"5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87\": container with ID starting with 5a39433e9519b6262ffb115763e69c8f05f0fac5478f91d0f0b5c1ecb72c4b87 not found: ID does not exist"
Sep 16 18:24:34.095054 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.095025    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59dcd55c64-nt9zc"]
Sep 16 18:24:34.098614 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.098588    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59dcd55c64-nt9zc"]
Sep 16 18:24:34.943944 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:24:34.943909    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="338c680b-fe77-44e7-8c7a-868682f99f49" path="/var/lib/kubelet/pods/338c680b-fe77-44e7-8c7a-868682f99f49/volumes"
Sep 16 18:25:38.299085 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.299054    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"]
Sep 16 18:25:38.299486 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.299414    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="338c680b-fe77-44e7-8c7a-868682f99f49" containerName="console"
Sep 16 18:25:38.299486 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.299426    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="338c680b-fe77-44e7-8c7a-868682f99f49" containerName="console"
Sep 16 18:25:38.299556 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.299494    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="338c680b-fe77-44e7-8c7a-868682f99f49" containerName="console"
Sep 16 18:25:38.302879 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.302864    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"
Sep 16 18:25:38.305272 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.305254    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 16 18:25:38.305397 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.305291    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 16 18:25:38.306033 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.306018    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-99mjw\""
Sep 16 18:25:38.311159 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.311141    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"]
Sep 16 18:25:38.352260 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.352237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98035573-4fa7-4f8a-a439-a12b929b9770-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dzvf8\" (UID: \"98035573-4fa7-4f8a-a439-a12b929b9770\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"
Sep 16 18:25:38.352370 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.352265    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfsbx\" (UniqueName: \"kubernetes.io/projected/98035573-4fa7-4f8a-a439-a12b929b9770-kube-api-access-xfsbx\") pod \"cert-manager-cainjector-559cdd8c87-dzvf8\" (UID: \"98035573-4fa7-4f8a-a439-a12b929b9770\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"
Sep 16 18:25:38.453650 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.453621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98035573-4fa7-4f8a-a439-a12b929b9770-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dzvf8\" (UID: \"98035573-4fa7-4f8a-a439-a12b929b9770\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"
Sep 16 18:25:38.453777 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.453654    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfsbx\" (UniqueName: \"kubernetes.io/projected/98035573-4fa7-4f8a-a439-a12b929b9770-kube-api-access-xfsbx\") pod \"cert-manager-cainjector-559cdd8c87-dzvf8\" (UID: \"98035573-4fa7-4f8a-a439-a12b929b9770\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"
Sep 16 18:25:38.461637 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.461609    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98035573-4fa7-4f8a-a439-a12b929b9770-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-dzvf8\" (UID: \"98035573-4fa7-4f8a-a439-a12b929b9770\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"
Sep 16 18:25:38.461775 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.461758    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfsbx\" (UniqueName: \"kubernetes.io/projected/98035573-4fa7-4f8a-a439-a12b929b9770-kube-api-access-xfsbx\") pod \"cert-manager-cainjector-559cdd8c87-dzvf8\" (UID: \"98035573-4fa7-4f8a-a439-a12b929b9770\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"
Sep 16 18:25:38.621316 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.621253    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"
Sep 16 18:25:38.889763 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:38.889740    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8"]
Sep 16 18:25:38.892143 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:25:38.892117    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98035573_4fa7_4f8a_a439_a12b929b9770.slice/crio-9ae6cb04e50f232a3cd0873a9f3c52f43fe340fc592145ee6973ea2f0ca761dc WatchSource:0}: Error finding container 9ae6cb04e50f232a3cd0873a9f3c52f43fe340fc592145ee6973ea2f0ca761dc: Status 404 returned error can't find the container with id 9ae6cb04e50f232a3cd0873a9f3c52f43fe340fc592145ee6973ea2f0ca761dc
Sep 16 18:25:39.341448 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:39.341412    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8" event={"ID":"98035573-4fa7-4f8a-a439-a12b929b9770","Type":"ContainerStarted","Data":"9ae6cb04e50f232a3cd0873a9f3c52f43fe340fc592145ee6973ea2f0ca761dc"}
Sep 16 18:25:42.356905 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:42.356810    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8" event={"ID":"98035573-4fa7-4f8a-a439-a12b929b9770","Type":"ContainerStarted","Data":"1a4f8f2787d771374e95b859f7216c83335d00ff014934a7b4200c7c1ddb7cf9"}
Sep 16 18:25:42.373361 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:42.373312    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-dzvf8" podStartSLOduration=1.267945809 podStartE2EDuration="4.373298564s" totalImagesPullingTime="3.105352755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:25:42.372475009 +0000 UTC m=+565.928768337" watchObservedRunningTime="2026-09-16 18:25:42.373298564 +0000 UTC m=+565.929591874"
Sep 16 18:25:50.726739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.726706    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"]
Sep 16 18:25:50.730045 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.730028    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"
Sep 16 18:25:50.733255 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.733237    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 18:25:50.733961 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.733948    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-5gsw6\""
Sep 16 18:25:50.734014 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.733967    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 18:25:50.754588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.754565    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"]
Sep 16 18:25:50.859416 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.859392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67717b12-c798-4df0-b7a5-41c0c9df16ca-tmp\") pod \"openshift-lws-operator-fd8ccff4c-29cmh\" (UID: \"67717b12-c798-4df0-b7a5-41c0c9df16ca\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"
Sep 16 18:25:50.859516 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.859442    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmqp\" (UniqueName: \"kubernetes.io/projected/67717b12-c798-4df0-b7a5-41c0c9df16ca-kube-api-access-5pmqp\") pod \"openshift-lws-operator-fd8ccff4c-29cmh\" (UID: \"67717b12-c798-4df0-b7a5-41c0c9df16ca\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"
Sep 16 18:25:50.960339 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.960311    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmqp\" (UniqueName: \"kubernetes.io/projected/67717b12-c798-4df0-b7a5-41c0c9df16ca-kube-api-access-5pmqp\") pod \"openshift-lws-operator-fd8ccff4c-29cmh\" (UID: \"67717b12-c798-4df0-b7a5-41c0c9df16ca\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"
Sep 16 18:25:50.960447 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.960373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67717b12-c798-4df0-b7a5-41c0c9df16ca-tmp\") pod \"openshift-lws-operator-fd8ccff4c-29cmh\" (UID: \"67717b12-c798-4df0-b7a5-41c0c9df16ca\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"
Sep 16 18:25:50.960689 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.960675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67717b12-c798-4df0-b7a5-41c0c9df16ca-tmp\") pod \"openshift-lws-operator-fd8ccff4c-29cmh\" (UID: \"67717b12-c798-4df0-b7a5-41c0c9df16ca\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"
Sep 16 18:25:50.972963 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:50.972943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmqp\" (UniqueName: \"kubernetes.io/projected/67717b12-c798-4df0-b7a5-41c0c9df16ca-kube-api-access-5pmqp\") pod \"openshift-lws-operator-fd8ccff4c-29cmh\" (UID: \"67717b12-c798-4df0-b7a5-41c0c9df16ca\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"
Sep 16 18:25:51.038168 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:51.038109    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"
Sep 16 18:25:51.314421 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:51.314398    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh"]
Sep 16 18:25:51.316937 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:25:51.316908    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67717b12_c798_4df0_b7a5_41c0c9df16ca.slice/crio-b94913040a8ab818e0f8e9f75c4a6601880e1a0a960799be4173ca7c416cfb60 WatchSource:0}: Error finding container b94913040a8ab818e0f8e9f75c4a6601880e1a0a960799be4173ca7c416cfb60: Status 404 returned error can't find the container with id b94913040a8ab818e0f8e9f75c4a6601880e1a0a960799be4173ca7c416cfb60
Sep 16 18:25:51.392138 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:51.392113    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh" event={"ID":"67717b12-c798-4df0-b7a5-41c0c9df16ca","Type":"ContainerStarted","Data":"b94913040a8ab818e0f8e9f75c4a6601880e1a0a960799be4173ca7c416cfb60"}
Sep 16 18:25:54.406504 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:54.406474    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh" event={"ID":"67717b12-c798-4df0-b7a5-41c0c9df16ca","Type":"ContainerStarted","Data":"56268ad8918082749b873fd3a4196e6963347ee364dc80210cd83d450c379936"}
Sep 16 18:25:54.426407 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:25:54.426362    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-29cmh" podStartSLOduration=1.512399104 podStartE2EDuration="4.426346448s" totalImagesPullingTime="2.913947344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:25:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:25:54.424553287 +0000 UTC m=+577.980846599" watchObservedRunningTime="2026-09-16 18:25:54.426346448 +0000 UTC m=+577.982639818"
Sep 16 18:26:07.547574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.547540    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"]
Sep 16 18:26:07.552760 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.552741    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.555360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.555335    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 18:26:07.555719 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.555706    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 18:26:07.556025 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.556013    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-dbp2n\""
Sep 16 18:26:07.556069 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.556057    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 18:26:07.561859 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.561838    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"]
Sep 16 18:26:07.592143 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.592115    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz478\" (UniqueName: \"kubernetes.io/projected/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-kube-api-access-nz478\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.592292 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.592158    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-metrics-cert\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.592292 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.592177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-manager-config\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.592407 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.592309    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-cert\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.692866 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.692841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-cert\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.693019 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.692876    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz478\" (UniqueName: \"kubernetes.io/projected/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-kube-api-access-nz478\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.693063 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.693013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-metrics-cert\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.693104 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.693064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-manager-config\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.693691 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.693671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-manager-config\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.695261 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.695242    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-cert\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.695366 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.695349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-metrics-cert\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.709995 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.709966    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz478\" (UniqueName: \"kubernetes.io/projected/b60c45ef-6b8a-4a85-b11c-dd3534c612ad-kube-api-access-nz478\") pod \"lws-controller-manager-5555c8c497-pkp5p\" (UID: \"b60c45ef-6b8a-4a85-b11c-dd3534c612ad\") " pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:07.863521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:07.863457    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:08.138725 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:08.138696    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"]
Sep 16 18:26:08.141427 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:26:08.141403    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60c45ef_6b8a_4a85_b11c_dd3534c612ad.slice/crio-c32ad791cf55003b0c1b594e672a97c0a36654b93b2738e03089b62c274a0d73 WatchSource:0}: Error finding container c32ad791cf55003b0c1b594e672a97c0a36654b93b2738e03089b62c274a0d73: Status 404 returned error can't find the container with id c32ad791cf55003b0c1b594e672a97c0a36654b93b2738e03089b62c274a0d73
Sep 16 18:26:08.467799 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:08.467763    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p" event={"ID":"b60c45ef-6b8a-4a85-b11c-dd3534c612ad","Type":"ContainerStarted","Data":"c32ad791cf55003b0c1b594e672a97c0a36654b93b2738e03089b62c274a0d73"}
Sep 16 18:26:13.490229 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:13.490128    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p" event={"ID":"b60c45ef-6b8a-4a85-b11c-dd3534c612ad","Type":"ContainerStarted","Data":"a552e72f35521c7b587afc3c312cdec15193edeba78906162066e839fef0f0ae"}
Sep 16 18:26:13.490564 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:13.490243    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:13.508418 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:13.508370    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p" podStartSLOduration=1.417101108 podStartE2EDuration="6.508353229s" totalImagesPullingTime="5.091252121s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:26:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:26:13.50683634 +0000 UTC m=+597.063129646" watchObservedRunningTime="2026-09-16 18:26:13.508353229 +0000 UTC m=+597.064646539"
Sep 16 18:26:19.012840 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.012807    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"]
Sep 16 18:26:19.016491 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.016470    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.018847 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.018827    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 18:26:19.018962 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.018910    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 18:26:19.019043 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.019022    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 18:26:19.019115 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.019062    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 18:26:19.019317 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.019300    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-r85q5\""
Sep 16 18:26:19.032681 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.032659    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"]
Sep 16 18:26:19.073122 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.073098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/56ecd90a-dc19-4d0a-aec4-edf4e7049414-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.073237 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.073124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56ecd90a-dc19-4d0a-aec4-edf4e7049414-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.073282 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.073244    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzvr2\" (UniqueName: \"kubernetes.io/projected/56ecd90a-dc19-4d0a-aec4-edf4e7049414-kube-api-access-pzvr2\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.073316 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.073280    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/56ecd90a-dc19-4d0a-aec4-edf4e7049414-manifests\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.073316 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.073298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56ecd90a-dc19-4d0a-aec4-edf4e7049414-tmp\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.174437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.174413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzvr2\" (UniqueName: \"kubernetes.io/projected/56ecd90a-dc19-4d0a-aec4-edf4e7049414-kube-api-access-pzvr2\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.174536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.174446    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/56ecd90a-dc19-4d0a-aec4-edf4e7049414-manifests\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.174536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.174466    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56ecd90a-dc19-4d0a-aec4-edf4e7049414-tmp\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.174536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.174491    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/56ecd90a-dc19-4d0a-aec4-edf4e7049414-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.174536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.174505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56ecd90a-dc19-4d0a-aec4-edf4e7049414-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.174840 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.174820    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/56ecd90a-dc19-4d0a-aec4-edf4e7049414-manifests\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.176654 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.176629    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56ecd90a-dc19-4d0a-aec4-edf4e7049414-tmp\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.176898 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.176882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56ecd90a-dc19-4d0a-aec4-edf4e7049414-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.176966 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.176924    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/56ecd90a-dc19-4d0a-aec4-edf4e7049414-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.181658 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.181637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzvr2\" (UniqueName: \"kubernetes.io/projected/56ecd90a-dc19-4d0a-aec4-edf4e7049414-kube-api-access-pzvr2\") pod \"opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz\" (UID: \"56ecd90a-dc19-4d0a-aec4-edf4e7049414\") " pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.326784 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.326724    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:19.601358 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:19.601333    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"]
Sep 16 18:26:19.602785 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:26:19.602754    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56ecd90a_dc19_4d0a_aec4_edf4e7049414.slice/crio-0b4d144b03b000a607dce34ae3ff492ecb1c81aab78d17b154877f41e22438b4 WatchSource:0}: Error finding container 0b4d144b03b000a607dce34ae3ff492ecb1c81aab78d17b154877f41e22438b4: Status 404 returned error can't find the container with id 0b4d144b03b000a607dce34ae3ff492ecb1c81aab78d17b154877f41e22438b4
Sep 16 18:26:20.525882 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:20.525840    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz" event={"ID":"56ecd90a-dc19-4d0a-aec4-edf4e7049414","Type":"ContainerStarted","Data":"0b4d144b03b000a607dce34ae3ff492ecb1c81aab78d17b154877f41e22438b4"}
Sep 16 18:26:22.536091 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:22.536059    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz" event={"ID":"56ecd90a-dc19-4d0a-aec4-edf4e7049414","Type":"ContainerStarted","Data":"db7ffb34b6fcdeb4307f666c7984f68318993407876201cf33431e6365b01b14"}
Sep 16 18:26:24.496860 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:24.496830    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5555c8c497-pkp5p"
Sep 16 18:26:24.546513 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:24.546482    2576 generic.go:356] "Generic (PLEG): container finished" podID="56ecd90a-dc19-4d0a-aec4-edf4e7049414" containerID="db7ffb34b6fcdeb4307f666c7984f68318993407876201cf33431e6365b01b14" exitCode=0
Sep 16 18:26:24.546672 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:24.546556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz" event={"ID":"56ecd90a-dc19-4d0a-aec4-edf4e7049414","Type":"ContainerDied","Data":"db7ffb34b6fcdeb4307f666c7984f68318993407876201cf33431e6365b01b14"}
Sep 16 18:26:25.552909 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:25.552875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz" event={"ID":"56ecd90a-dc19-4d0a-aec4-edf4e7049414","Type":"ContainerStarted","Data":"a4009287c855e26847c5c3378e09b9d5752c7c5343a56cc6bb323a0791f5d2cb"}
Sep 16 18:26:25.553398 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:25.553090    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:25.573848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:25.573803    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz" podStartSLOduration=3.314325287 podStartE2EDuration="7.57378976s" totalImagesPullingTime="3.249692372s" totalInitContainerRuntime="1.009772101s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:26:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:26:25.57278728 +0000 UTC m=+609.129080594" watchObservedRunningTime="2026-09-16 18:26:25.57378976 +0000 UTC m=+609.130083071"
Sep 16 18:26:36.559966 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:36.559934    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz"
Sep 16 18:26:39.614881 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.614804    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"]
Sep 16 18:26:39.622612 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.622593    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.625313 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.625282    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 16 18:26:39.625426 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.625294    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 18:26:39.625884 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.625867    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 16 18:26:39.626144 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.626128    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tklbp\""
Sep 16 18:26:39.626264 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.626143    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 18:26:39.634080 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.634060    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"]
Sep 16 18:26:39.760857 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.760829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c283f1b-8108-44fe-8e78-2e5c3fa572fe-tmp\") pod \"kube-auth-proxy-79f4c98848-jnvlv\" (UID: \"4c283f1b-8108-44fe-8e78-2e5c3fa572fe\") " pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.761000 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.760890    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2snl\" (UniqueName: \"kubernetes.io/projected/4c283f1b-8108-44fe-8e78-2e5c3fa572fe-kube-api-access-z2snl\") pod \"kube-auth-proxy-79f4c98848-jnvlv\" (UID: \"4c283f1b-8108-44fe-8e78-2e5c3fa572fe\") " pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.761000 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.760957    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c283f1b-8108-44fe-8e78-2e5c3fa572fe-tls-certs\") pod \"kube-auth-proxy-79f4c98848-jnvlv\" (UID: \"4c283f1b-8108-44fe-8e78-2e5c3fa572fe\") " pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.862247 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.862201    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2snl\" (UniqueName: \"kubernetes.io/projected/4c283f1b-8108-44fe-8e78-2e5c3fa572fe-kube-api-access-z2snl\") pod \"kube-auth-proxy-79f4c98848-jnvlv\" (UID: \"4c283f1b-8108-44fe-8e78-2e5c3fa572fe\") " pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.862382 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.862263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c283f1b-8108-44fe-8e78-2e5c3fa572fe-tls-certs\") pod \"kube-auth-proxy-79f4c98848-jnvlv\" (UID: \"4c283f1b-8108-44fe-8e78-2e5c3fa572fe\") " pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.862382 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.862285    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c283f1b-8108-44fe-8e78-2e5c3fa572fe-tmp\") pod \"kube-auth-proxy-79f4c98848-jnvlv\" (UID: \"4c283f1b-8108-44fe-8e78-2e5c3fa572fe\") " pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.864607 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.864575    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c283f1b-8108-44fe-8e78-2e5c3fa572fe-tmp\") pod \"kube-auth-proxy-79f4c98848-jnvlv\" (UID: \"4c283f1b-8108-44fe-8e78-2e5c3fa572fe\") " pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.864784 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.864765    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c283f1b-8108-44fe-8e78-2e5c3fa572fe-tls-certs\") pod \"kube-auth-proxy-79f4c98848-jnvlv\" (UID: \"4c283f1b-8108-44fe-8e78-2e5c3fa572fe\") " pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.871838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.871786    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2snl\" (UniqueName: \"kubernetes.io/projected/4c283f1b-8108-44fe-8e78-2e5c3fa572fe-kube-api-access-z2snl\") pod \"kube-auth-proxy-79f4c98848-jnvlv\" (UID: \"4c283f1b-8108-44fe-8e78-2e5c3fa572fe\") " pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:39.932707 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:39.932687    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"
Sep 16 18:26:40.226916 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:40.226887    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv"]
Sep 16 18:26:40.230801 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:26:40.230781    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c283f1b_8108_44fe_8e78_2e5c3fa572fe.slice/crio-c47d7ec7c4ad1cf2a10f28d8f33f6c18de8f925251f688b9cf48c0d6defb0a16 WatchSource:0}: Error finding container c47d7ec7c4ad1cf2a10f28d8f33f6c18de8f925251f688b9cf48c0d6defb0a16: Status 404 returned error can't find the container with id c47d7ec7c4ad1cf2a10f28d8f33f6c18de8f925251f688b9cf48c0d6defb0a16
Sep 16 18:26:40.633640 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:40.633562    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv" event={"ID":"4c283f1b-8108-44fe-8e78-2e5c3fa572fe","Type":"ContainerStarted","Data":"c47d7ec7c4ad1cf2a10f28d8f33f6c18de8f925251f688b9cf48c0d6defb0a16"}
Sep 16 18:26:43.651248 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:43.651186    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv" event={"ID":"4c283f1b-8108-44fe-8e78-2e5c3fa572fe","Type":"ContainerStarted","Data":"08485577f201cf2ed05208c14eceabec4f37e8c826fb9e08a6206b6935e80d57"}
Sep 16 18:26:43.667546 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:26:43.667488    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv" podStartSLOduration=1.981055671 podStartE2EDuration="4.667471948s" totalImagesPullingTime="2.686416277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:26:43.666178701 +0000 UTC m=+627.222472012" watchObservedRunningTime="2026-09-16 18:26:43.667471948 +0000 UTC m=+627.223765258"
Sep 16 18:27:32.597888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.597852    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56cb87b9bf-bjstk"]
Sep 16 18:27:32.601973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.601954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.611600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.611579    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56cb87b9bf-bjstk"]
Sep 16 18:27:32.722975 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.722943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-oauth-serving-cert\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.722975 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.722980    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-service-ca\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.723241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.723005    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e9ffea6-8a9d-4315-99f5-648c8b919b07-console-oauth-config\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.723241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.723061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-trusted-ca-bundle\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.723241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.723132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-console-config\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.723241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.723181    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nshth\" (UniqueName: \"kubernetes.io/projected/5e9ffea6-8a9d-4315-99f5-648c8b919b07-kube-api-access-nshth\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.723241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.723234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e9ffea6-8a9d-4315-99f5-648c8b919b07-console-serving-cert\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.824435 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.824402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e9ffea6-8a9d-4315-99f5-648c8b919b07-console-serving-cert\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.824435 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.824438    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-oauth-serving-cert\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.824662 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.824467    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-service-ca\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.824662 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.824500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e9ffea6-8a9d-4315-99f5-648c8b919b07-console-oauth-config\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.824662 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.824523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-trusted-ca-bundle\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.824662 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.824559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-console-config\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.824662 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.824633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nshth\" (UniqueName: \"kubernetes.io/projected/5e9ffea6-8a9d-4315-99f5-648c8b919b07-kube-api-access-nshth\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.825296 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.825268    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-oauth-serving-cert\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.825436 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.825346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-service-ca\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.825436 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.825381    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-trusted-ca-bundle\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.825436 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.825407    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e9ffea6-8a9d-4315-99f5-648c8b919b07-console-config\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.826997 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.826973    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e9ffea6-8a9d-4315-99f5-648c8b919b07-console-serving-cert\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.826997 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.826991    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e9ffea6-8a9d-4315-99f5-648c8b919b07-console-oauth-config\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.832406 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.832386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nshth\" (UniqueName: \"kubernetes.io/projected/5e9ffea6-8a9d-4315-99f5-648c8b919b07-kube-api-access-nshth\") pod \"console-56cb87b9bf-bjstk\" (UID: \"5e9ffea6-8a9d-4315-99f5-648c8b919b07\") " pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:32.911834 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:32.911809    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:33.187111 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:33.187083    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56cb87b9bf-bjstk"]
Sep 16 18:27:33.188933 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:27:33.188903    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e9ffea6_8a9d_4315_99f5_648c8b919b07.slice/crio-812665e7ffc60c1a57a8878d849def69b2054f1ab271658c77680de40e82d927 WatchSource:0}: Error finding container 812665e7ffc60c1a57a8878d849def69b2054f1ab271658c77680de40e82d927: Status 404 returned error can't find the container with id 812665e7ffc60c1a57a8878d849def69b2054f1ab271658c77680de40e82d927
Sep 16 18:27:33.872287 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:33.872249    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cb87b9bf-bjstk" event={"ID":"5e9ffea6-8a9d-4315-99f5-648c8b919b07","Type":"ContainerStarted","Data":"8337c4209b74deb7e99934a328c0c83d44e47d82630aca8ae657754462f6acfd"}
Sep 16 18:27:33.872661 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:33.872304    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56cb87b9bf-bjstk" event={"ID":"5e9ffea6-8a9d-4315-99f5-648c8b919b07","Type":"ContainerStarted","Data":"812665e7ffc60c1a57a8878d849def69b2054f1ab271658c77680de40e82d927"}
Sep 16 18:27:33.892658 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:33.892612    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56cb87b9bf-bjstk" podStartSLOduration=1.892596374 podStartE2EDuration="1.892596374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:27:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:27:33.891231914 +0000 UTC m=+677.447525224" watchObservedRunningTime="2026-09-16 18:27:33.892596374 +0000 UTC m=+677.448889685"
Sep 16 18:27:42.911959 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:42.911925    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:42.912373 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:42.911982    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:42.916593 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:42.916573    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:43.921128 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:43.921096    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56cb87b9bf-bjstk"
Sep 16 18:27:43.965772 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:27:43.965739    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cff88886-5rmd8"]
Sep 16 18:28:08.991978 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:08.991917    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cff88886-5rmd8" podUID="c3039d09-6266-4ebf-a1ad-7b10780bd812" containerName="console" containerID="cri-o://f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec" gracePeriod=15
Sep 16 18:28:09.235021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.234996    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:28:09.341176 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341111    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-trusted-ca-bundle\") pod \"c3039d09-6266-4ebf-a1ad-7b10780bd812\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") "
Sep 16 18:28:09.341176 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341154    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zxh9\" (UniqueName: \"kubernetes.io/projected/c3039d09-6266-4ebf-a1ad-7b10780bd812-kube-api-access-6zxh9\") pod \"c3039d09-6266-4ebf-a1ad-7b10780bd812\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") "
Sep 16 18:28:09.341385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341195    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-service-ca\") pod \"c3039d09-6266-4ebf-a1ad-7b10780bd812\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") "
Sep 16 18:28:09.341385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341247    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-oauth-config\") pod \"c3039d09-6266-4ebf-a1ad-7b10780bd812\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") "
Sep 16 18:28:09.341385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341265    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-oauth-serving-cert\") pod \"c3039d09-6266-4ebf-a1ad-7b10780bd812\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") "
Sep 16 18:28:09.341385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341285    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-config\") pod \"c3039d09-6266-4ebf-a1ad-7b10780bd812\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") "
Sep 16 18:28:09.341385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341346    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-serving-cert\") pod \"c3039d09-6266-4ebf-a1ad-7b10780bd812\" (UID: \"c3039d09-6266-4ebf-a1ad-7b10780bd812\") "
Sep 16 18:28:09.341742 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341695    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-service-ca" (OuterVolumeSpecName: "service-ca") pod "c3039d09-6266-4ebf-a1ad-7b10780bd812" (UID: "c3039d09-6266-4ebf-a1ad-7b10780bd812"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:28:09.341893 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341768    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-config" (OuterVolumeSpecName: "console-config") pod "c3039d09-6266-4ebf-a1ad-7b10780bd812" (UID: "c3039d09-6266-4ebf-a1ad-7b10780bd812"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:28:09.341893 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341782    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c3039d09-6266-4ebf-a1ad-7b10780bd812" (UID: "c3039d09-6266-4ebf-a1ad-7b10780bd812"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:28:09.341893 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.341799    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c3039d09-6266-4ebf-a1ad-7b10780bd812" (UID: "c3039d09-6266-4ebf-a1ad-7b10780bd812"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:28:09.343375 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.343340    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c3039d09-6266-4ebf-a1ad-7b10780bd812" (UID: "c3039d09-6266-4ebf-a1ad-7b10780bd812"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:28:09.343474 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.343405    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c3039d09-6266-4ebf-a1ad-7b10780bd812" (UID: "c3039d09-6266-4ebf-a1ad-7b10780bd812"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:28:09.343474 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.343417    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3039d09-6266-4ebf-a1ad-7b10780bd812-kube-api-access-6zxh9" (OuterVolumeSpecName: "kube-api-access-6zxh9") pod "c3039d09-6266-4ebf-a1ad-7b10780bd812" (UID: "c3039d09-6266-4ebf-a1ad-7b10780bd812"). InnerVolumeSpecName "kube-api-access-6zxh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:28:09.442647 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.442625    2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-oauth-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:28:09.442647 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.442647    2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:28:09.442775 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.442657    2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-serving-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:28:09.442775 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.442667    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-trusted-ca-bundle\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:28:09.442775 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.442675    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zxh9\" (UniqueName: \"kubernetes.io/projected/c3039d09-6266-4ebf-a1ad-7b10780bd812-kube-api-access-6zxh9\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:28:09.442775 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.442684    2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3039d09-6266-4ebf-a1ad-7b10780bd812-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:28:09.442775 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:09.442691    2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3039d09-6266-4ebf-a1ad-7b10780bd812-console-oauth-config\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:28:10.034031 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.033995    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3039d09-6266-4ebf-a1ad-7b10780bd812" containerID="f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec" exitCode=0
Sep 16 18:28:10.034455 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.034038    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cff88886-5rmd8"
Sep 16 18:28:10.034455 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.034058    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cff88886-5rmd8" event={"ID":"c3039d09-6266-4ebf-a1ad-7b10780bd812","Type":"ContainerDied","Data":"f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec"}
Sep 16 18:28:10.034455 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.034106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cff88886-5rmd8" event={"ID":"c3039d09-6266-4ebf-a1ad-7b10780bd812","Type":"ContainerDied","Data":"dd652a7f0c68534054f2acd1c71b6c4552a05df31beaa0dadb0e9444183b2b3b"}
Sep 16 18:28:10.034455 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.034131    2576 scope.go:117] "RemoveContainer" containerID="f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec"
Sep 16 18:28:10.043182 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.043163    2576 scope.go:117] "RemoveContainer" containerID="f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec"
Sep 16 18:28:10.043480 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:28:10.043458    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec\": container with ID starting with f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec not found: ID does not exist" containerID="f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec"
Sep 16 18:28:10.043526 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.043491    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec"} err="failed to get container status \"f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec\": rpc error: code = NotFound desc = could not find container \"f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec\": container with ID starting with f7a143db546ca0d9b0445e193a2c709d7370629a8913b62b55e58c7d177230ec not found: ID does not exist"
Sep 16 18:28:10.058946 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.058917    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cff88886-5rmd8"]
Sep 16 18:28:10.062812 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.062791    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cff88886-5rmd8"]
Sep 16 18:28:10.944595 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:10.944564    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3039d09-6266-4ebf-a1ad-7b10780bd812" path="/var/lib/kubelet/pods/c3039d09-6266-4ebf-a1ad-7b10780bd812/volumes"
Sep 16 18:28:24.962570 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.962537    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"]
Sep 16 18:28:24.962931 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.962867    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3039d09-6266-4ebf-a1ad-7b10780bd812" containerName="console"
Sep 16 18:28:24.962931 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.962877    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3039d09-6266-4ebf-a1ad-7b10780bd812" containerName="console"
Sep 16 18:28:24.963001 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.962945    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3039d09-6266-4ebf-a1ad-7b10780bd812" containerName="console"
Sep 16 18:28:24.967030 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.967014    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:24.969326 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.969299    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 16 18:28:24.969491 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.969472    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 16 18:28:24.969687 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.969668    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 16 18:28:24.970052 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.970040    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 16 18:28:24.970097 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.970061    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4l2gg\""
Sep 16 18:28:24.974909 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:24.974889    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"]
Sep 16 18:28:25.084126 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.084096    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4bb689b-f7a2-4982-81cd-371f2cfc6823-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6qbxp\" (UID: \"a4bb689b-f7a2-4982-81cd-371f2cfc6823\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.084309 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.084139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhkl5\" (UniqueName: \"kubernetes.io/projected/a4bb689b-f7a2-4982-81cd-371f2cfc6823-kube-api-access-fhkl5\") pod \"kuadrant-console-plugin-fbdd4f445-6qbxp\" (UID: \"a4bb689b-f7a2-4982-81cd-371f2cfc6823\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.084309 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.084165    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a4bb689b-f7a2-4982-81cd-371f2cfc6823-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-6qbxp\" (UID: \"a4bb689b-f7a2-4982-81cd-371f2cfc6823\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.185396 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.185364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4bb689b-f7a2-4982-81cd-371f2cfc6823-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6qbxp\" (UID: \"a4bb689b-f7a2-4982-81cd-371f2cfc6823\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.185584 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.185414    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhkl5\" (UniqueName: \"kubernetes.io/projected/a4bb689b-f7a2-4982-81cd-371f2cfc6823-kube-api-access-fhkl5\") pod \"kuadrant-console-plugin-fbdd4f445-6qbxp\" (UID: \"a4bb689b-f7a2-4982-81cd-371f2cfc6823\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.185584 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.185528    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a4bb689b-f7a2-4982-81cd-371f2cfc6823-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-6qbxp\" (UID: \"a4bb689b-f7a2-4982-81cd-371f2cfc6823\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.186121 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.186096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a4bb689b-f7a2-4982-81cd-371f2cfc6823-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-6qbxp\" (UID: \"a4bb689b-f7a2-4982-81cd-371f2cfc6823\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.187776 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.187759    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4bb689b-f7a2-4982-81cd-371f2cfc6823-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-6qbxp\" (UID: \"a4bb689b-f7a2-4982-81cd-371f2cfc6823\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.193681 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.193654    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhkl5\" (UniqueName: \"kubernetes.io/projected/a4bb689b-f7a2-4982-81cd-371f2cfc6823-kube-api-access-fhkl5\") pod \"kuadrant-console-plugin-fbdd4f445-6qbxp\" (UID: \"a4bb689b-f7a2-4982-81cd-371f2cfc6823\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.277787 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.277716    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"
Sep 16 18:28:25.548819 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.548794    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp"]
Sep 16 18:28:25.551123 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:28:25.551090    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4bb689b_f7a2_4982_81cd_371f2cfc6823.slice/crio-c9fe0ab56c0490937d8caf2863328eea4e70506e5c607f6d2ccabf37837d420b WatchSource:0}: Error finding container c9fe0ab56c0490937d8caf2863328eea4e70506e5c607f6d2ccabf37837d420b: Status 404 returned error can't find the container with id c9fe0ab56c0490937d8caf2863328eea4e70506e5c607f6d2ccabf37837d420b
Sep 16 18:28:25.552755 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:25.552736    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:28:26.105899 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:26.105853    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp" event={"ID":"a4bb689b-f7a2-4982-81cd-371f2cfc6823","Type":"ContainerStarted","Data":"c9fe0ab56c0490937d8caf2863328eea4e70506e5c607f6d2ccabf37837d420b"}
Sep 16 18:28:31.135076 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:31.135040    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp" event={"ID":"a4bb689b-f7a2-4982-81cd-371f2cfc6823","Type":"ContainerStarted","Data":"1b3da455da26b14bfd899e775d400fa482f10822c4d2507cb24add2f9061775c"}
Sep 16 18:28:31.152696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:31.152646    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-6qbxp" podStartSLOduration=2.470500799 podStartE2EDuration="7.152631047s" totalImagesPullingTime="4.682130248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:31.151203346 +0000 UTC m=+734.707496658" watchObservedRunningTime="2026-09-16 18:28:31.152631047 +0000 UTC m=+734.708924357"
Sep 16 18:28:35.099199 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.099168    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"]
Sep 16 18:28:35.103906 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.103886    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:35.106135 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.106115    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dfm9g\""
Sep 16 18:28:35.114178 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.114153    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"]
Sep 16 18:28:35.168622 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.168598    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jtqm\" (UniqueName: \"kubernetes.io/projected/fabb4a19-92f1-4f2e-aad9-098b25892ede-kube-api-access-2jtqm\") pod \"kuadrant-operator-controller-manager-5f685c496f-6k72k\" (UID: \"fabb4a19-92f1-4f2e-aad9-098b25892ede\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:35.168727 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.168651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fabb4a19-92f1-4f2e-aad9-098b25892ede-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-6k72k\" (UID: \"fabb4a19-92f1-4f2e-aad9-098b25892ede\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:35.269555 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.269529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jtqm\" (UniqueName: \"kubernetes.io/projected/fabb4a19-92f1-4f2e-aad9-098b25892ede-kube-api-access-2jtqm\") pod \"kuadrant-operator-controller-manager-5f685c496f-6k72k\" (UID: \"fabb4a19-92f1-4f2e-aad9-098b25892ede\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:35.269677 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.269575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fabb4a19-92f1-4f2e-aad9-098b25892ede-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-6k72k\" (UID: \"fabb4a19-92f1-4f2e-aad9-098b25892ede\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:35.269896 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.269882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fabb4a19-92f1-4f2e-aad9-098b25892ede-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-6k72k\" (UID: \"fabb4a19-92f1-4f2e-aad9-098b25892ede\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:35.277539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.277511    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jtqm\" (UniqueName: \"kubernetes.io/projected/fabb4a19-92f1-4f2e-aad9-098b25892ede-kube-api-access-2jtqm\") pod \"kuadrant-operator-controller-manager-5f685c496f-6k72k\" (UID: \"fabb4a19-92f1-4f2e-aad9-098b25892ede\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:35.414315 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.414295    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:35.692008 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.691980    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"]
Sep 16 18:28:35.694512 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:28:35.694483    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfabb4a19_92f1_4f2e_aad9_098b25892ede.slice/crio-a70a9873c971fcbd05cd594abe9570dac2026344ce0db67970cbf637b71de018 WatchSource:0}: Error finding container a70a9873c971fcbd05cd594abe9570dac2026344ce0db67970cbf637b71de018: Status 404 returned error can't find the container with id a70a9873c971fcbd05cd594abe9570dac2026344ce0db67970cbf637b71de018
Sep 16 18:28:35.899487 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.899453    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"]
Sep 16 18:28:35.903880 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.903850    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"]
Sep 16 18:28:35.925819 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.925792    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"]
Sep 16 18:28:35.930434 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.930412    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:28:35.944120 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.944068    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"]
Sep 16 18:28:35.977182 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.977152    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbsxp\" (UniqueName: \"kubernetes.io/projected/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-kube-api-access-pbsxp\") pod \"kuadrant-operator-controller-manager-5f685c496f-ptvp6\" (UID: \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:28:35.977380 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:35.977330    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-ptvp6\" (UID: \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:28:36.078069 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:36.078037    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-ptvp6\" (UID: \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:28:36.078240 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:36.078093    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbsxp\" (UniqueName: \"kubernetes.io/projected/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-kube-api-access-pbsxp\") pod \"kuadrant-operator-controller-manager-5f685c496f-ptvp6\" (UID: \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:28:36.078445 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:36.078427    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-ptvp6\" (UID: \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:28:36.087876 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:36.087854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbsxp\" (UniqueName: \"kubernetes.io/projected/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-kube-api-access-pbsxp\") pod \"kuadrant-operator-controller-manager-5f685c496f-ptvp6\" (UID: \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:28:36.241548 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:36.241471    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:28:36.524369 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:36.524330    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"]
Sep 16 18:28:36.527960 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:28:36.527922    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4412d1_95fb_4581_b473_dd8a1fa4f13c.slice/crio-c04161b8647d2579b6f193a0ba1c29dbc65be9ad673fc2bafc5768f98d4c4313 WatchSource:0}: Error finding container c04161b8647d2579b6f193a0ba1c29dbc65be9ad673fc2bafc5768f98d4c4313: Status 404 returned error can't find the container with id c04161b8647d2579b6f193a0ba1c29dbc65be9ad673fc2bafc5768f98d4c4313
Sep 16 18:28:37.179291 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:37.179253    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6" event={"ID":"fd4412d1-95fb-4581-b473-dd8a1fa4f13c","Type":"ContainerStarted","Data":"c04161b8647d2579b6f193a0ba1c29dbc65be9ad673fc2bafc5768f98d4c4313"}
Sep 16 18:28:40.205387 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.205342    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k" podUID="fabb4a19-92f1-4f2e-aad9-098b25892ede" containerName="manager" containerID="cri-o://feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2" gracePeriod=2
Sep 16 18:28:40.207149 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.207120    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6" event={"ID":"fd4412d1-95fb-4581-b473-dd8a1fa4f13c","Type":"ContainerStarted","Data":"1a12f00ec3cc57b86b164e80e34635e06e8c9726d7254fa54f7a298c6ea264b9"}
Sep 16 18:28:40.207281 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.207248    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:28:40.232496 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.232444    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6" podStartSLOduration=1.6976037590000002 podStartE2EDuration="5.232428773s" totalImagesPullingTime="3.534825014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:28:40.230713493 +0000 UTC m=+743.787006805" watchObservedRunningTime="2026-09-16 18:28:40.232428773 +0000 UTC m=+743.788722085"
Sep 16 18:28:40.440620 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.440594    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:40.443273 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:28:40.443246    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-6k72k\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="fabb4a19-92f1-4f2e-aad9-098b25892ede" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:40.521757 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.521734    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fabb4a19-92f1-4f2e-aad9-098b25892ede-extensions-socket-volume\") pod \"fabb4a19-92f1-4f2e-aad9-098b25892ede\" (UID: \"fabb4a19-92f1-4f2e-aad9-098b25892ede\") "
Sep 16 18:28:40.521878 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.521811    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jtqm\" (UniqueName: \"kubernetes.io/projected/fabb4a19-92f1-4f2e-aad9-098b25892ede-kube-api-access-2jtqm\") pod \"fabb4a19-92f1-4f2e-aad9-098b25892ede\" (UID: \"fabb4a19-92f1-4f2e-aad9-098b25892ede\") "
Sep 16 18:28:40.522021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.521996    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fabb4a19-92f1-4f2e-aad9-098b25892ede-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fabb4a19-92f1-4f2e-aad9-098b25892ede" (UID: "fabb4a19-92f1-4f2e-aad9-098b25892ede"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:28:40.522078 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.522067    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fabb4a19-92f1-4f2e-aad9-098b25892ede-extensions-socket-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:28:40.523864 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.523842    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fabb4a19-92f1-4f2e-aad9-098b25892ede-kube-api-access-2jtqm" (OuterVolumeSpecName: "kube-api-access-2jtqm") pod "fabb4a19-92f1-4f2e-aad9-098b25892ede" (UID: "fabb4a19-92f1-4f2e-aad9-098b25892ede"). InnerVolumeSpecName "kube-api-access-2jtqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:28:40.623478 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.623419    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jtqm\" (UniqueName: \"kubernetes.io/projected/fabb4a19-92f1-4f2e-aad9-098b25892ede-kube-api-access-2jtqm\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:28:40.944161 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:40.944131    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fabb4a19-92f1-4f2e-aad9-098b25892ede" path="/var/lib/kubelet/pods/fabb4a19-92f1-4f2e-aad9-098b25892ede/volumes"
Sep 16 18:28:41.213398 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:41.213316    2576 generic.go:356] "Generic (PLEG): container finished" podID="fabb4a19-92f1-4f2e-aad9-098b25892ede" containerID="feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2" exitCode=2
Sep 16 18:28:41.213398 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:41.213364    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:41.213398 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:41.213373    2576 scope.go:117] "RemoveContainer" containerID="feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2"
Sep 16 18:28:41.217677 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:28:41.217650    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-6k72k\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="fabb4a19-92f1-4f2e-aad9-098b25892ede" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-6k72k"
Sep 16 18:28:41.221691 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:41.221674    2576 scope.go:117] "RemoveContainer" containerID="feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2"
Sep 16 18:28:41.221945 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:28:41.221927    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2\": container with ID starting with feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2 not found: ID does not exist" containerID="feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2"
Sep 16 18:28:41.222026 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:41.221957    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2"} err="failed to get container status \"feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2\": rpc error: code = NotFound desc = could not find container \"feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2\": container with ID starting with feaa237aa3367a4c0737ce056d2b3bd7587608369e2736e19eca2f0f3324a1d2 not found: ID does not exist"
Sep 16 18:28:51.216179 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:28:51.216149    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:29:04.040431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:04.040394    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"]
Sep 16 18:29:04.040933 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:04.040585    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6" podUID="fd4412d1-95fb-4581-b473-dd8a1fa4f13c" containerName="manager" containerID="cri-o://1a12f00ec3cc57b86b164e80e34635e06e8c9726d7254fa54f7a298c6ea264b9" gracePeriod=10
Sep 16 18:29:04.320705 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:04.320648    2576 generic.go:356] "Generic (PLEG): container finished" podID="fd4412d1-95fb-4581-b473-dd8a1fa4f13c" containerID="1a12f00ec3cc57b86b164e80e34635e06e8c9726d7254fa54f7a298c6ea264b9" exitCode=0
Sep 16 18:29:04.320806 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:04.320719    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6" event={"ID":"fd4412d1-95fb-4581-b473-dd8a1fa4f13c","Type":"ContainerDied","Data":"1a12f00ec3cc57b86b164e80e34635e06e8c9726d7254fa54f7a298c6ea264b9"}
Sep 16 18:29:04.815733 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:04.815707    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:29:04.936069 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:04.936006    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbsxp\" (UniqueName: \"kubernetes.io/projected/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-kube-api-access-pbsxp\") pod \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\" (UID: \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\") "
Sep 16 18:29:04.936198 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:04.936117    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-extensions-socket-volume\") pod \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\" (UID: \"fd4412d1-95fb-4581-b473-dd8a1fa4f13c\") "
Sep 16 18:29:04.936618 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:04.936592    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fd4412d1-95fb-4581-b473-dd8a1fa4f13c" (UID: "fd4412d1-95fb-4581-b473-dd8a1fa4f13c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:29:04.937988 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:04.937968    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-kube-api-access-pbsxp" (OuterVolumeSpecName: "kube-api-access-pbsxp") pod "fd4412d1-95fb-4581-b473-dd8a1fa4f13c" (UID: "fd4412d1-95fb-4581-b473-dd8a1fa4f13c"). InnerVolumeSpecName "kube-api-access-pbsxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:29:05.036801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:05.036782    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-extensions-socket-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:29:05.036801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:05.036803    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbsxp\" (UniqueName: \"kubernetes.io/projected/fd4412d1-95fb-4581-b473-dd8a1fa4f13c-kube-api-access-pbsxp\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:29:05.327265 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:05.327178    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"
Sep 16 18:29:05.327265 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:05.327175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6" event={"ID":"fd4412d1-95fb-4581-b473-dd8a1fa4f13c","Type":"ContainerDied","Data":"c04161b8647d2579b6f193a0ba1c29dbc65be9ad673fc2bafc5768f98d4c4313"}
Sep 16 18:29:05.327676 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:05.327270    2576 scope.go:117] "RemoveContainer" containerID="1a12f00ec3cc57b86b164e80e34635e06e8c9726d7254fa54f7a298c6ea264b9"
Sep 16 18:29:05.381496 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:05.381473    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"]
Sep 16 18:29:05.388550 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:05.388529    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-ptvp6"]
Sep 16 18:29:06.951472 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:06.951440    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd4412d1-95fb-4581-b473-dd8a1fa4f13c" path="/var/lib/kubelet/pods/fd4412d1-95fb-4581-b473-dd8a1fa4f13c/volumes"
Sep 16 18:29:23.299380 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.299346    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7dxgr"]
Sep 16 18:29:23.299734 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.299694    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fabb4a19-92f1-4f2e-aad9-098b25892ede" containerName="manager"
Sep 16 18:29:23.299734 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.299705    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fabb4a19-92f1-4f2e-aad9-098b25892ede" containerName="manager"
Sep 16 18:29:23.299734 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.299724    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd4412d1-95fb-4581-b473-dd8a1fa4f13c" containerName="manager"
Sep 16 18:29:23.299734 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.299729    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd4412d1-95fb-4581-b473-dd8a1fa4f13c" containerName="manager"
Sep 16 18:29:23.299863 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.299784    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd4412d1-95fb-4581-b473-dd8a1fa4f13c" containerName="manager"
Sep 16 18:29:23.299863 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.299792    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fabb4a19-92f1-4f2e-aad9-098b25892ede" containerName="manager"
Sep 16 18:29:23.304085 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.304070    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:23.306411 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.306392    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 16 18:29:23.311142 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.311114    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7dxgr"]
Sep 16 18:29:23.392430 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.392394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-config-file\") pod \"limitador-limitador-8588749977-7dxgr\" (UID: \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\") " pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:23.392583 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.392483    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krlkm\" (UniqueName: \"kubernetes.io/projected/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-kube-api-access-krlkm\") pod \"limitador-limitador-8588749977-7dxgr\" (UID: \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\") " pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:23.396894 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.396860    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7dxgr"]
Sep 16 18:29:23.493552 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.493523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-config-file\") pod \"limitador-limitador-8588749977-7dxgr\" (UID: \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\") " pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:23.493690 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.493565    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krlkm\" (UniqueName: \"kubernetes.io/projected/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-kube-api-access-krlkm\") pod \"limitador-limitador-8588749977-7dxgr\" (UID: \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\") " pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:23.494165 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.494147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-config-file\") pod \"limitador-limitador-8588749977-7dxgr\" (UID: \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\") " pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:23.502115 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.502096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krlkm\" (UniqueName: \"kubernetes.io/projected/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-kube-api-access-krlkm\") pod \"limitador-limitador-8588749977-7dxgr\" (UID: \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\") " pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:23.614946 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.614880    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:23.882423 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:23.882396    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7dxgr"]
Sep 16 18:29:23.884400 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:29:23.884372    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c39b3d8_6ed5_43e7_9000_e0dfaf9aa773.slice/crio-5a7945cf25de53451a07129bad73c96e0b1180b6a573d3a1a9f44297352f8ab2 WatchSource:0}: Error finding container 5a7945cf25de53451a07129bad73c96e0b1180b6a573d3a1a9f44297352f8ab2: Status 404 returned error can't find the container with id 5a7945cf25de53451a07129bad73c96e0b1180b6a573d3a1a9f44297352f8ab2
Sep 16 18:29:24.114454 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.114426    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5bh8b"]
Sep 16 18:29:24.118773 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.118754    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b"
Sep 16 18:29:24.120727 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.120710    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h6z7c\""
Sep 16 18:29:24.125251 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.125230    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5bh8b"]
Sep 16 18:29:24.200591 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.200565    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsh4s\" (UniqueName: \"kubernetes.io/projected/eef4f0e1-9953-49b9-ab23-cdda8e2c451e-kube-api-access-fsh4s\") pod \"authorino-7bcfd8f9cd-5bh8b\" (UID: \"eef4f0e1-9953-49b9-ab23-cdda8e2c451e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b"
Sep 16 18:29:24.301470 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.301442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsh4s\" (UniqueName: \"kubernetes.io/projected/eef4f0e1-9953-49b9-ab23-cdda8e2c451e-kube-api-access-fsh4s\") pod \"authorino-7bcfd8f9cd-5bh8b\" (UID: \"eef4f0e1-9953-49b9-ab23-cdda8e2c451e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b"
Sep 16 18:29:24.309124 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.309101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsh4s\" (UniqueName: \"kubernetes.io/projected/eef4f0e1-9953-49b9-ab23-cdda8e2c451e-kube-api-access-fsh4s\") pod \"authorino-7bcfd8f9cd-5bh8b\" (UID: \"eef4f0e1-9953-49b9-ab23-cdda8e2c451e\") " pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b"
Sep 16 18:29:24.416699 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.416666    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr" event={"ID":"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773","Type":"ContainerStarted","Data":"5a7945cf25de53451a07129bad73c96e0b1180b6a573d3a1a9f44297352f8ab2"}
Sep 16 18:29:24.429286 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.429250    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b"
Sep 16 18:29:24.700995 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:24.700969    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5bh8b"]
Sep 16 18:29:24.703621 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:29:24.703592    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeef4f0e1_9953_49b9_ab23_cdda8e2c451e.slice/crio-e64af405c8466ac9afec86bf0a5dcc7d207526d1cc3896a3bb2a7b52a04e829a WatchSource:0}: Error finding container e64af405c8466ac9afec86bf0a5dcc7d207526d1cc3896a3bb2a7b52a04e829a: Status 404 returned error can't find the container with id e64af405c8466ac9afec86bf0a5dcc7d207526d1cc3896a3bb2a7b52a04e829a
Sep 16 18:29:25.422251 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:25.422202    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr" event={"ID":"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773","Type":"ContainerStarted","Data":"71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc"}
Sep 16 18:29:25.422595 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:25.422277    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:25.423395 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:25.423376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b" event={"ID":"eef4f0e1-9953-49b9-ab23-cdda8e2c451e","Type":"ContainerStarted","Data":"e64af405c8466ac9afec86bf0a5dcc7d207526d1cc3896a3bb2a7b52a04e829a"}
Sep 16 18:29:25.444343 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:25.444305    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr" podStartSLOduration=1.026739614 podStartE2EDuration="2.444291582s" totalImagesPullingTime="1.417551968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:25.442602471 +0000 UTC m=+788.998895803" watchObservedRunningTime="2026-09-16 18:29:25.444291582 +0000 UTC m=+789.000584893"
Sep 16 18:29:27.221729 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:27.221694    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5bh8b"]
Sep 16 18:29:27.434504 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:27.434469    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b" event={"ID":"eef4f0e1-9953-49b9-ab23-cdda8e2c451e","Type":"ContainerStarted","Data":"2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31"}
Sep 16 18:29:27.453100 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:27.453051    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b" podStartSLOduration=1.320521687 podStartE2EDuration="3.453035445s" totalImagesPullingTime="2.132513758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:27.452090454 +0000 UTC m=+791.008383791" watchObservedRunningTime="2026-09-16 18:29:27.453035445 +0000 UTC m=+791.009328755"
Sep 16 18:29:28.439094 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:28.439059    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b" podUID="eef4f0e1-9953-49b9-ab23-cdda8e2c451e" containerName="authorino" containerID="cri-o://2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31" gracePeriod=30
Sep 16 18:29:28.682874 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:28.682850    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b"
Sep 16 18:29:28.739468 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:28.739409    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fsh4s\" (UniqueName: \"kubernetes.io/projected/eef4f0e1-9953-49b9-ab23-cdda8e2c451e-kube-api-access-fsh4s\") pod \"eef4f0e1-9953-49b9-ab23-cdda8e2c451e\" (UID: \"eef4f0e1-9953-49b9-ab23-cdda8e2c451e\") "
Sep 16 18:29:28.741288 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:28.741267    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eef4f0e1-9953-49b9-ab23-cdda8e2c451e-kube-api-access-fsh4s" (OuterVolumeSpecName: "kube-api-access-fsh4s") pod "eef4f0e1-9953-49b9-ab23-cdda8e2c451e" (UID: "eef4f0e1-9953-49b9-ab23-cdda8e2c451e"). InnerVolumeSpecName "kube-api-access-fsh4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:29:28.840533 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:28.840508    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsh4s\" (UniqueName: \"kubernetes.io/projected/eef4f0e1-9953-49b9-ab23-cdda8e2c451e-kube-api-access-fsh4s\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:29:29.444920 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:29.444884    2576 generic.go:356] "Generic (PLEG): container finished" podID="eef4f0e1-9953-49b9-ab23-cdda8e2c451e" containerID="2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31" exitCode=0
Sep 16 18:29:29.445371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:29.444925    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b"
Sep 16 18:29:29.445371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:29.444926    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b" event={"ID":"eef4f0e1-9953-49b9-ab23-cdda8e2c451e","Type":"ContainerDied","Data":"2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31"}
Sep 16 18:29:29.445371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:29.444974    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-5bh8b" event={"ID":"eef4f0e1-9953-49b9-ab23-cdda8e2c451e","Type":"ContainerDied","Data":"e64af405c8466ac9afec86bf0a5dcc7d207526d1cc3896a3bb2a7b52a04e829a"}
Sep 16 18:29:29.445371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:29.444998    2576 scope.go:117] "RemoveContainer" containerID="2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31"
Sep 16 18:29:29.453332 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:29.453303    2576 scope.go:117] "RemoveContainer" containerID="2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31"
Sep 16 18:29:29.453606 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:29:29.453581    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31\": container with ID starting with 2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31 not found: ID does not exist" containerID="2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31"
Sep 16 18:29:29.453671 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:29.453614    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31"} err="failed to get container status \"2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31\": rpc error: code = NotFound desc = could not find container \"2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31\": container with ID starting with 2d61f5a0d76a38ae6ab1a2fec1f7562de73e3adc1a29daaa4c5ab758330d7b31 not found: ID does not exist"
Sep 16 18:29:29.471153 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:29.471130    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5bh8b"]
Sep 16 18:29:29.475119 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:29.475101    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-5bh8b"]
Sep 16 18:29:30.944142 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:30.944112    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eef4f0e1-9953-49b9-ab23-cdda8e2c451e" path="/var/lib/kubelet/pods/eef4f0e1-9953-49b9-ab23-cdda8e2c451e/volumes"
Sep 16 18:29:36.430085 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:36.430059    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:37.235609 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:37.235580    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7dxgr"]
Sep 16 18:29:37.235777 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:37.235754    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr" podUID="4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773" containerName="limitador" containerID="cri-o://71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc" gracePeriod=30
Sep 16 18:29:38.176028 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.176002    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:38.315420 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.315345    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-config-file\") pod \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\" (UID: \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\") "
Sep 16 18:29:38.315420 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.315390    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krlkm\" (UniqueName: \"kubernetes.io/projected/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-kube-api-access-krlkm\") pod \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\" (UID: \"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773\") "
Sep 16 18:29:38.315662 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.315639    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-config-file" (OuterVolumeSpecName: "config-file") pod "4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773" (UID: "4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:29:38.317307 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.317284    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-kube-api-access-krlkm" (OuterVolumeSpecName: "kube-api-access-krlkm") pod "4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773" (UID: "4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773"). InnerVolumeSpecName "kube-api-access-krlkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:29:38.416345 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.416317    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-config-file\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:29:38.416345 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.416337    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krlkm\" (UniqueName: \"kubernetes.io/projected/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773-kube-api-access-krlkm\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:29:38.488635 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.488603    2576 generic.go:356] "Generic (PLEG): container finished" podID="4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773" containerID="71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc" exitCode=0
Sep 16 18:29:38.488755 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.488649    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr"
Sep 16 18:29:38.488755 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.488692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr" event={"ID":"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773","Type":"ContainerDied","Data":"71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc"}
Sep 16 18:29:38.488755 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.488747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-7dxgr" event={"ID":"4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773","Type":"ContainerDied","Data":"5a7945cf25de53451a07129bad73c96e0b1180b6a573d3a1a9f44297352f8ab2"}
Sep 16 18:29:38.488857 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.488774    2576 scope.go:117] "RemoveContainer" containerID="71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc"
Sep 16 18:29:38.497601 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.497584    2576 scope.go:117] "RemoveContainer" containerID="71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc"
Sep 16 18:29:38.497842 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:29:38.497825    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc\": container with ID starting with 71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc not found: ID does not exist" containerID="71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc"
Sep 16 18:29:38.497892 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.497851    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc"} err="failed to get container status \"71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc\": rpc error: code = NotFound desc = could not find container \"71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc\": container with ID starting with 71872abff5fd6cc679f535dd3b62dd5ad13444db3af492402798e51b40f119cc not found: ID does not exist"
Sep 16 18:29:38.510042 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.510021    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7dxgr"]
Sep 16 18:29:38.513163 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.513145    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-7dxgr"]
Sep 16 18:29:38.943995 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:38.943956    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773" path="/var/lib/kubelet/pods/4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773/volumes"
Sep 16 18:29:42.021894 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.021853    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"]
Sep 16 18:29:42.022352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.022258    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773" containerName="limitador"
Sep 16 18:29:42.022352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.022289    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773" containerName="limitador"
Sep 16 18:29:42.022352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.022321    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eef4f0e1-9953-49b9-ab23-cdda8e2c451e" containerName="authorino"
Sep 16 18:29:42.022352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.022327    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eef4f0e1-9953-49b9-ab23-cdda8e2c451e" containerName="authorino"
Sep 16 18:29:42.022560 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.022407    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c39b3d8-6ed5-43e7-9000-e0dfaf9aa773" containerName="limitador"
Sep 16 18:29:42.022560 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.022415    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eef4f0e1-9953-49b9-ab23-cdda8e2c451e" containerName="authorino"
Sep 16 18:29:42.026728 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.026709    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:29:42.029390 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.029368    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 18:29:42.029495 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.029401    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 16 18:29:42.029707 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.029681    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 16 18:29:42.029890 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.029869    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-bxddq\""
Sep 16 18:29:42.031006 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.030987    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"]
Sep 16 18:29:42.145232 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.145171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e4500109-c468-4163-a946-d0ce59c9de06-data\") pod \"postgres-69fb4594cf-lx6ks\" (UID: \"e4500109-c468-4163-a946-d0ce59c9de06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:29:42.145377 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.145316    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czw9z\" (UniqueName: \"kubernetes.io/projected/e4500109-c468-4163-a946-d0ce59c9de06-kube-api-access-czw9z\") pod \"postgres-69fb4594cf-lx6ks\" (UID: \"e4500109-c468-4163-a946-d0ce59c9de06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:29:42.246197 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.246175    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czw9z\" (UniqueName: \"kubernetes.io/projected/e4500109-c468-4163-a946-d0ce59c9de06-kube-api-access-czw9z\") pod \"postgres-69fb4594cf-lx6ks\" (UID: \"e4500109-c468-4163-a946-d0ce59c9de06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:29:42.246329 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.246236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e4500109-c468-4163-a946-d0ce59c9de06-data\") pod \"postgres-69fb4594cf-lx6ks\" (UID: \"e4500109-c468-4163-a946-d0ce59c9de06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:29:42.246528 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.246513    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e4500109-c468-4163-a946-d0ce59c9de06-data\") pod \"postgres-69fb4594cf-lx6ks\" (UID: \"e4500109-c468-4163-a946-d0ce59c9de06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:29:42.254618 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.254594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czw9z\" (UniqueName: \"kubernetes.io/projected/e4500109-c468-4163-a946-d0ce59c9de06-kube-api-access-czw9z\") pod \"postgres-69fb4594cf-lx6ks\" (UID: \"e4500109-c468-4163-a946-d0ce59c9de06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:29:42.338352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.338302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:29:42.604156 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:42.604134    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"]
Sep 16 18:29:42.606572 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:29:42.606545    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4500109_c468_4163_a946_d0ce59c9de06.slice/crio-9017737b11370d099f33d3029f5fefdd9499051a923a1daffb19145f0953d4ca WatchSource:0}: Error finding container 9017737b11370d099f33d3029f5fefdd9499051a923a1daffb19145f0953d4ca: Status 404 returned error can't find the container with id 9017737b11370d099f33d3029f5fefdd9499051a923a1daffb19145f0953d4ca
Sep 16 18:29:43.515462 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:43.515420    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks" event={"ID":"e4500109-c468-4163-a946-d0ce59c9de06","Type":"ContainerStarted","Data":"9017737b11370d099f33d3029f5fefdd9499051a923a1daffb19145f0953d4ca"}
Sep 16 18:29:47.547453 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:47.547363    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks" event={"ID":"e4500109-c468-4163-a946-d0ce59c9de06","Type":"ContainerStarted","Data":"6be74bd4ce184e867f5de3222bf053c32a35bfb9669e89a02b203ea60f677c54"}
Sep 16 18:29:47.547820 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:47.547482    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:29:47.564263 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:47.564191    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks" podStartSLOduration=0.912081937 podStartE2EDuration="5.564174376s" totalImagesPullingTime="4.652092439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:29:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:29:47.562844733 +0000 UTC m=+811.119138041" watchObservedRunningTime="2026-09-16 18:29:47.564174376 +0000 UTC m=+811.120467689"
Sep 16 18:29:53.580563 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:29:53.580528    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lx6ks"
Sep 16 18:30:01.047320 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.047288    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-88b7ff4dd-srkcq"]
Sep 16 18:30:01.064981 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.064962    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.068916 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.068890    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 18:30:01.069095 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.069069    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 18:30:01.069660 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.069640    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 18:30:01.069891 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.069665    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n9qfx\""
Sep 16 18:30:01.071435 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.071412    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-88b7ff4dd-srkcq"]
Sep 16 18:30:01.210752 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.210723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wshs7\" (UniqueName: \"kubernetes.io/projected/c5785f36-fc74-4585-93c4-9deafb8cb2b0-kube-api-access-wshs7\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.210865 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.210767    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-metrics-certs\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.210865 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.210842    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-webhook-cert\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.234226 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.234185    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c46454c74-4t6h9"]
Sep 16 18:30:01.237061 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.237044    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.249527 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.249509    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c46454c74-4t6h9"]
Sep 16 18:30:01.311652 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.311593    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-webhook-cert\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.311652 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.311634    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wshs7\" (UniqueName: \"kubernetes.io/projected/c5785f36-fc74-4585-93c4-9deafb8cb2b0-kube-api-access-wshs7\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.311781 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.311677    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-metrics-certs\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.311781 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:30:01.311739    2576 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Sep 16 18:30:01.311854 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:30:01.311798    2576 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 16 18:30:01.311854 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:30:01.311801    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-webhook-cert podName:c5785f36-fc74-4585-93c4-9deafb8cb2b0 nodeName:}" failed. No retries permitted until 2026-09-16 18:30:01.811781811 +0000 UTC m=+825.368075117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-webhook-cert") pod "maas-controller-88b7ff4dd-srkcq" (UID: "c5785f36-fc74-4585-93c4-9deafb8cb2b0") : secret "maas-controller-webhook-cert" not found
Sep 16 18:30:01.311967 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:30:01.311866    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-metrics-certs podName:c5785f36-fc74-4585-93c4-9deafb8cb2b0 nodeName:}" failed. No retries permitted until 2026-09-16 18:30:01.811854297 +0000 UTC m=+825.368147589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-metrics-certs") pod "maas-controller-88b7ff4dd-srkcq" (UID: "c5785f36-fc74-4585-93c4-9deafb8cb2b0") : secret "maas-controller-metrics-tls" not found
Sep 16 18:30:01.322127 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.322104    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wshs7\" (UniqueName: \"kubernetes.io/projected/c5785f36-fc74-4585-93c4-9deafb8cb2b0-kube-api-access-wshs7\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.412862 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.412834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-webhook-cert\") pod \"maas-controller-6c46454c74-4t6h9\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") " pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.412862 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.412865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-metrics-certs\") pod \"maas-controller-6c46454c74-4t6h9\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") " pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.413005 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.412935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsjdm\" (UniqueName: \"kubernetes.io/projected/159f0984-3a84-424b-a94b-f477ea325155-kube-api-access-dsjdm\") pod \"maas-controller-6c46454c74-4t6h9\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") " pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.513590 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.513566    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-webhook-cert\") pod \"maas-controller-6c46454c74-4t6h9\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") " pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.513690 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.513592    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-metrics-certs\") pod \"maas-controller-6c46454c74-4t6h9\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") " pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.513803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.513783    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsjdm\" (UniqueName: \"kubernetes.io/projected/159f0984-3a84-424b-a94b-f477ea325155-kube-api-access-dsjdm\") pod \"maas-controller-6c46454c74-4t6h9\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") " pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.516009 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.515987    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-metrics-certs\") pod \"maas-controller-6c46454c74-4t6h9\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") " pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.516099 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.516041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-webhook-cert\") pod \"maas-controller-6c46454c74-4t6h9\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") " pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.521251 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.521235    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsjdm\" (UniqueName: \"kubernetes.io/projected/159f0984-3a84-424b-a94b-f477ea325155-kube-api-access-dsjdm\") pod \"maas-controller-6c46454c74-4t6h9\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") " pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.546113 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.546088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:01.816011 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.815983    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-metrics-certs\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.816326 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.816075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-webhook-cert\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.817092 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.817072    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c46454c74-4t6h9"]
Sep 16 18:30:01.818612 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.818580    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-webhook-cert\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.818724 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.818593    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-metrics-certs\") pod \"maas-controller-88b7ff4dd-srkcq\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") " pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:01.819085 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:30:01.819065    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod159f0984_3a84_424b_a94b_f477ea325155.slice/crio-65b791b286b8890a8dba42ad67b42534bdaa67ad5ed8888c2d20760c4d5981c0 WatchSource:0}: Error finding container 65b791b286b8890a8dba42ad67b42534bdaa67ad5ed8888c2d20760c4d5981c0: Status 404 returned error can't find the container with id 65b791b286b8890a8dba42ad67b42534bdaa67ad5ed8888c2d20760c4d5981c0
Sep 16 18:30:01.976653 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:01.976625    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:02.245238 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:02.245189    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-88b7ff4dd-srkcq"]
Sep 16 18:30:02.247203 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:30:02.247162    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5785f36_fc74_4585_93c4_9deafb8cb2b0.slice/crio-be84df074854738427c0dd306ebb3e414cfe056f13fa66783d77a81b884c6a10 WatchSource:0}: Error finding container be84df074854738427c0dd306ebb3e414cfe056f13fa66783d77a81b884c6a10: Status 404 returned error can't find the container with id be84df074854738427c0dd306ebb3e414cfe056f13fa66783d77a81b884c6a10
Sep 16 18:30:02.622154 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:02.622071    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88b7ff4dd-srkcq" event={"ID":"c5785f36-fc74-4585-93c4-9deafb8cb2b0","Type":"ContainerStarted","Data":"be84df074854738427c0dd306ebb3e414cfe056f13fa66783d77a81b884c6a10"}
Sep 16 18:30:02.623252 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:02.623231    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-4t6h9" event={"ID":"159f0984-3a84-424b-a94b-f477ea325155","Type":"ContainerStarted","Data":"65b791b286b8890a8dba42ad67b42534bdaa67ad5ed8888c2d20760c4d5981c0"}
Sep 16 18:30:05.641361 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:05.641326    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88b7ff4dd-srkcq" event={"ID":"c5785f36-fc74-4585-93c4-9deafb8cb2b0","Type":"ContainerStarted","Data":"09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c"}
Sep 16 18:30:05.641763 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:05.641451    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:05.642919 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:05.642898    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-4t6h9" event={"ID":"159f0984-3a84-424b-a94b-f477ea325155","Type":"ContainerStarted","Data":"e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739"}
Sep 16 18:30:05.643039 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:05.643011    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:05.660411 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:05.660357    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-88b7ff4dd-srkcq" podStartSLOduration=2.061299259 podStartE2EDuration="4.66034423s" totalImagesPullingTime="2.599044971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:05.65932515 +0000 UTC m=+829.215618464" watchObservedRunningTime="2026-09-16 18:30:05.66034423 +0000 UTC m=+829.216637540"
Sep 16 18:30:05.679574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:05.679536    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c46454c74-4t6h9" podStartSLOduration=1.6415434119999999 podStartE2EDuration="4.679523877s" totalImagesPullingTime="3.037980465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:05.678236323 +0000 UTC m=+829.234529654" watchObservedRunningTime="2026-09-16 18:30:05.679523877 +0000 UTC m=+829.235817248"
Sep 16 18:30:16.649887 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:16.649859    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:30:16.650279 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:16.649925    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:16.715537 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:16.715506    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-88b7ff4dd-srkcq"]
Sep 16 18:30:16.715713 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:16.715691    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-88b7ff4dd-srkcq" podUID="c5785f36-fc74-4585-93c4-9deafb8cb2b0" containerName="manager" containerID="cri-o://09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c" gracePeriod=30
Sep 16 18:30:16.974261 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:16.974238    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:17.055551 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.055524    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wshs7\" (UniqueName: \"kubernetes.io/projected/c5785f36-fc74-4585-93c4-9deafb8cb2b0-kube-api-access-wshs7\") pod \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") "
Sep 16 18:30:17.055738 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.055569    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-metrics-certs\") pod \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") "
Sep 16 18:30:17.055738 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.055612    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-webhook-cert\") pod \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\" (UID: \"c5785f36-fc74-4585-93c4-9deafb8cb2b0\") "
Sep 16 18:30:17.057905 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.057871    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c5785f36-fc74-4585-93c4-9deafb8cb2b0" (UID: "c5785f36-fc74-4585-93c4-9deafb8cb2b0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:17.058008 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.057906    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c5785f36-fc74-4585-93c4-9deafb8cb2b0" (UID: "c5785f36-fc74-4585-93c4-9deafb8cb2b0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:30:17.058008 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.057911    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5785f36-fc74-4585-93c4-9deafb8cb2b0-kube-api-access-wshs7" (OuterVolumeSpecName: "kube-api-access-wshs7") pod "c5785f36-fc74-4585-93c4-9deafb8cb2b0" (UID: "c5785f36-fc74-4585-93c4-9deafb8cb2b0"). InnerVolumeSpecName "kube-api-access-wshs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:30:17.157028 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.156999    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wshs7\" (UniqueName: \"kubernetes.io/projected/c5785f36-fc74-4585-93c4-9deafb8cb2b0-kube-api-access-wshs7\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:30:17.157028 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.157026    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:30:17.157028 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.157035    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c5785f36-fc74-4585-93c4-9deafb8cb2b0-webhook-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:30:17.700636 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.700602    2576 generic.go:356] "Generic (PLEG): container finished" podID="c5785f36-fc74-4585-93c4-9deafb8cb2b0" containerID="09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c" exitCode=0
Sep 16 18:30:17.701113 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.700656    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88b7ff4dd-srkcq"
Sep 16 18:30:17.701113 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.700665    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88b7ff4dd-srkcq" event={"ID":"c5785f36-fc74-4585-93c4-9deafb8cb2b0","Type":"ContainerDied","Data":"09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c"}
Sep 16 18:30:17.701113 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.700717    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88b7ff4dd-srkcq" event={"ID":"c5785f36-fc74-4585-93c4-9deafb8cb2b0","Type":"ContainerDied","Data":"be84df074854738427c0dd306ebb3e414cfe056f13fa66783d77a81b884c6a10"}
Sep 16 18:30:17.701113 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.700749    2576 scope.go:117] "RemoveContainer" containerID="09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c"
Sep 16 18:30:17.709917 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.709899    2576 scope.go:117] "RemoveContainer" containerID="09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c"
Sep 16 18:30:17.710171 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:30:17.710153    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c\": container with ID starting with 09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c not found: ID does not exist" containerID="09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c"
Sep 16 18:30:17.710276 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.710180    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c"} err="failed to get container status \"09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c\": rpc error: code = NotFound desc = could not find container \"09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c\": container with ID starting with 09bf65524bc864de87b5b4d96495d0f70070a9311eaf65a32da779eb555e810c not found: ID does not exist"
Sep 16 18:30:17.720737 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.720716    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-88b7ff4dd-srkcq"]
Sep 16 18:30:17.724120 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:17.724098    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-88b7ff4dd-srkcq"]
Sep 16 18:30:18.944398 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:18.944366    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5785f36-fc74-4585-93c4-9deafb8cb2b0" path="/var/lib/kubelet/pods/c5785f36-fc74-4585-93c4-9deafb8cb2b0/volumes"
Sep 16 18:30:39.798935 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.798899    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"]
Sep 16 18:30:39.799432 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.799299    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5785f36-fc74-4585-93c4-9deafb8cb2b0" containerName="manager"
Sep 16 18:30:39.799432 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.799313    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5785f36-fc74-4585-93c4-9deafb8cb2b0" containerName="manager"
Sep 16 18:30:39.799432 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.799386    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5785f36-fc74-4585-93c4-9deafb8cb2b0" containerName="manager"
Sep 16 18:30:39.803725 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.803704    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:39.808007 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.807981    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 18:30:39.808139 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.808115    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-54t9j\""
Sep 16 18:30:39.820487 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.820462    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"]
Sep 16 18:30:39.965139 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.965106    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xf8n\" (UniqueName: \"kubernetes.io/projected/d1aad725-09fe-4095-ab39-57f3aad985f1-kube-api-access-5xf8n\") pod \"payload-processing-5b54b7fbb7-vr6p8\" (UID: \"d1aad725-09fe-4095-ab39-57f3aad985f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:39.965313 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:39.965185    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1aad725-09fe-4095-ab39-57f3aad985f1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-vr6p8\" (UID: \"d1aad725-09fe-4095-ab39-57f3aad985f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:40.066354 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:40.066271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xf8n\" (UniqueName: \"kubernetes.io/projected/d1aad725-09fe-4095-ab39-57f3aad985f1-kube-api-access-5xf8n\") pod \"payload-processing-5b54b7fbb7-vr6p8\" (UID: \"d1aad725-09fe-4095-ab39-57f3aad985f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:40.066509 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:40.066362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1aad725-09fe-4095-ab39-57f3aad985f1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-vr6p8\" (UID: \"d1aad725-09fe-4095-ab39-57f3aad985f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:40.067265 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:40.067235    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1aad725-09fe-4095-ab39-57f3aad985f1-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-vr6p8\" (UID: \"d1aad725-09fe-4095-ab39-57f3aad985f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:40.075538 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:40.075514    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xf8n\" (UniqueName: \"kubernetes.io/projected/d1aad725-09fe-4095-ab39-57f3aad985f1-kube-api-access-5xf8n\") pod \"payload-processing-5b54b7fbb7-vr6p8\" (UID: \"d1aad725-09fe-4095-ab39-57f3aad985f1\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:40.113685 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:40.113657    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:40.381348 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:40.381323    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"]
Sep 16 18:30:40.383320 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:30:40.383278    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1aad725_09fe_4095_ab39_57f3aad985f1.slice/crio-3a14e45f1207612b7baea4c199033f37ff47932bc86f76b5ef1f4fe4649f5d0e WatchSource:0}: Error finding container 3a14e45f1207612b7baea4c199033f37ff47932bc86f76b5ef1f4fe4649f5d0e: Status 404 returned error can't find the container with id 3a14e45f1207612b7baea4c199033f37ff47932bc86f76b5ef1f4fe4649f5d0e
Sep 16 18:30:40.819476 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:40.819445    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8" event={"ID":"d1aad725-09fe-4095-ab39-57f3aad985f1","Type":"ContainerStarted","Data":"3a14e45f1207612b7baea4c199033f37ff47932bc86f76b5ef1f4fe4649f5d0e"}
Sep 16 18:30:42.830907 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:42.830794    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8" event={"ID":"d1aad725-09fe-4095-ab39-57f3aad985f1","Type":"ContainerStarted","Data":"142b8b894804188f16c6c51d9d962af21dd811cd7812e2bd0c03c72f4de9b71c"}
Sep 16 18:30:42.831385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:42.830930    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:42.848248 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:42.848187    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8" podStartSLOduration=1.731927711 podStartE2EDuration="3.848172395s" totalImagesPullingTime="2.116244684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:42.846431204 +0000 UTC m=+866.402724515" watchObservedRunningTime="2026-09-16 18:30:42.848172395 +0000 UTC m=+866.404465705"
Sep 16 18:30:53.838392 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:53.838359    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-vr6p8"
Sep 16 18:30:57.926879 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:57.926850    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"]
Sep 16 18:30:57.930381 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:57.930366    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:57.936239 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:57.936202    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5zsbs\""
Sep 16 18:30:57.936378 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:57.936242    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 16 18:30:57.936378 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:57.936266    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 18:30:57.937080 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:57.937060    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 18:30:57.944814 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:57.944793    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"]
Sep 16 18:30:58.018218 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.018189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e1f211-7bf4-4f72-b474-464563ce15c3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.018360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.018245    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.018360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.018264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.018360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.018335    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.018464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.018362    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7n5j\" (UniqueName: \"kubernetes.io/projected/a3e1f211-7bf4-4f72-b474-464563ce15c3-kube-api-access-z7n5j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.018464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.018387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.119507 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.119473    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.119507 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.119508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7n5j\" (UniqueName: \"kubernetes.io/projected/a3e1f211-7bf4-4f72-b474-464563ce15c3-kube-api-access-z7n5j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.119738 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.119533    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.119738 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.119587    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e1f211-7bf4-4f72-b474-464563ce15c3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.119738 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.119607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.119738 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.119622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.119975 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.119864    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.119975 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.119930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.120073 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.119997    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.121861 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.121843    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3e1f211-7bf4-4f72-b474-464563ce15c3-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.122150 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.122131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3e1f211-7bf4-4f72-b474-464563ce15c3-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.128046 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.128020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7n5j\" (UniqueName: \"kubernetes.io/projected/a3e1f211-7bf4-4f72-b474-464563ce15c3-kube-api-access-z7n5j\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc\" (UID: \"a3e1f211-7bf4-4f72-b474-464563ce15c3\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.240731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.240652    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:30:58.512928 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.512855    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"]
Sep 16 18:30:58.515158 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:30:58.515110    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e1f211_7bf4_4f72_b474_464563ce15c3.slice/crio-1d44bdb98a0797312ab1b5ea615a9bd7f5191e8df2dabcb34bdb8f9e260cb192 WatchSource:0}: Error finding container 1d44bdb98a0797312ab1b5ea615a9bd7f5191e8df2dabcb34bdb8f9e260cb192: Status 404 returned error can't find the container with id 1d44bdb98a0797312ab1b5ea615a9bd7f5191e8df2dabcb34bdb8f9e260cb192
Sep 16 18:30:58.910288 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:58.910245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc" event={"ID":"a3e1f211-7bf4-4f72-b474-464563ce15c3","Type":"ContainerStarted","Data":"1d44bdb98a0797312ab1b5ea615a9bd7f5191e8df2dabcb34bdb8f9e260cb192"}
Sep 16 18:30:59.916850 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:59.916813    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc" event={"ID":"a3e1f211-7bf4-4f72-b474-464563ce15c3","Type":"ContainerStarted","Data":"30c590af5aa1285da4014ab2d69bd1b7ec6f1109176d888deec1ec06e9528eed"}
Sep 16 18:30:59.936430 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:30:59.936382    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc" podStartSLOduration=1.9277530139999999 podStartE2EDuration="2.936365525s" totalImagesPullingTime="1.008612511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:30:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:30:59.934606571 +0000 UTC m=+883.490899882" watchObservedRunningTime="2026-09-16 18:30:59.936365525 +0000 UTC m=+883.492658836"
Sep 16 18:31:08.240823 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:08.240784    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:31:08.241228 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:08.240837    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:31:08.253285 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:08.253261    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:31:08.974289 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:08.974258    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc"
Sep 16 18:31:37.937750 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.937719    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"]
Sep 16 18:31:37.943430 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.943406    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:37.946540 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.946518    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 16 18:31:37.959557 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.959536    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"]
Sep 16 18:31:37.981428 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.981408    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsfmk\" (UniqueName: \"kubernetes.io/projected/bb3ae5ca-1cd1-4668-967d-633957149360-kube-api-access-qsfmk\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:37.981546 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.981450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:37.981546 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.981520    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3ae5ca-1cd1-4668-967d-633957149360-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:37.981669 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.981596    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:37.981669 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.981629    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:37.981669 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:37.981653    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.082659 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.082612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.082821 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.082676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.082821 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.082712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.082821 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.082783    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsfmk\" (UniqueName: \"kubernetes.io/projected/bb3ae5ca-1cd1-4668-967d-633957149360-kube-api-access-qsfmk\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.082990 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.082851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.082990 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.082906    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3ae5ca-1cd1-4668-967d-633957149360-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.083099 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.082988    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.083099 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.083072    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.083244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.083137    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.085035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.085013    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bb3ae5ca-1cd1-4668-967d-633957149360-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.085332 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.085315    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bb3ae5ca-1cd1-4668-967d-633957149360-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.095322 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.095295    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsfmk\" (UniqueName: \"kubernetes.io/projected/bb3ae5ca-1cd1-4668-967d-633957149360-kube-api-access-qsfmk\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf\" (UID: \"bb3ae5ca-1cd1-4668-967d-633957149360\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.253598 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.253541    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:38.534642 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:38.534424    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"]
Sep 16 18:31:38.537243 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:31:38.537192    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3ae5ca_1cd1_4668_967d_633957149360.slice/crio-069620d18d12e09b5c15a50544a522900a5c784de3d553f8e81dba9e9fb1bced WatchSource:0}: Error finding container 069620d18d12e09b5c15a50544a522900a5c784de3d553f8e81dba9e9fb1bced: Status 404 returned error can't find the container with id 069620d18d12e09b5c15a50544a522900a5c784de3d553f8e81dba9e9fb1bced
Sep 16 18:31:39.103626 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:39.103592    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf" event={"ID":"bb3ae5ca-1cd1-4668-967d-633957149360","Type":"ContainerStarted","Data":"4da78568da4120b454e35388407c24d4ba3a2cac76f6d87899009bdcb81e4cdd"}
Sep 16 18:31:39.103996 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:39.103644    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf" event={"ID":"bb3ae5ca-1cd1-4668-967d-633957149360","Type":"ContainerStarted","Data":"069620d18d12e09b5c15a50544a522900a5c784de3d553f8e81dba9e9fb1bced"}
Sep 16 18:31:39.127530 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:39.127482    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf" podStartSLOduration=1.935089201 podStartE2EDuration="2.127465682s" totalImagesPullingTime="192.376481ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:31:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:31:39.125716093 +0000 UTC m=+922.682009403" watchObservedRunningTime="2026-09-16 18:31:39.127465682 +0000 UTC m=+922.683758993"
Sep 16 18:31:48.254020 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:48.253986    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:48.254534 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:48.254039    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:48.266487 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:48.266466    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:31:49.162788 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:31:49.162760    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf"
Sep 16 18:32:06.158553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.158521    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"]
Sep 16 18:32:06.162400 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.162381    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.166032 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.166009    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 16 18:32:06.175474 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.175450    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"]
Sep 16 18:32:06.224732 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.224707    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4t9n\" (UniqueName: \"kubernetes.io/projected/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-kube-api-access-x4t9n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.224884 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.224768    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.224884 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.224840    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.224978 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.224901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.225047 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.225028    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.225122 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.225099    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.325775 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.325743    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.325914 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.325821    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.325914 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.325872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.325914 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.325912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.326095 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.325950    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4t9n\" (UniqueName: \"kubernetes.io/projected/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-kube-api-access-x4t9n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.326095 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.325987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.326204 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.326188    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.326286 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.326238    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.326326 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.326300    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.328227 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.328196    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.328444 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.328424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.337006 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.336980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4t9n\" (UniqueName: \"kubernetes.io/projected/10d1faf7-a952-473d-bb0c-f5b3c16b53fa-kube-api-access-x4t9n\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m\" (UID: \"10d1faf7-a952-473d-bb0c-f5b3c16b53fa\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.473030 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.472961    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:06.752466 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:06.752378    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"]
Sep 16 18:32:06.754757 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:32:06.754729    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10d1faf7_a952_473d_bb0c_f5b3c16b53fa.slice/crio-1c4c0f243350d99ffee52904f9b5bfb3d0722dc6b503d0619c50ce64df1d36d8 WatchSource:0}: Error finding container 1c4c0f243350d99ffee52904f9b5bfb3d0722dc6b503d0619c50ce64df1d36d8: Status 404 returned error can't find the container with id 1c4c0f243350d99ffee52904f9b5bfb3d0722dc6b503d0619c50ce64df1d36d8
Sep 16 18:32:07.236658 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:07.236624    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m" event={"ID":"10d1faf7-a952-473d-bb0c-f5b3c16b53fa","Type":"ContainerStarted","Data":"bd1a49a0fca2452f5dc99856e469c94262bf798ffd22f5e27d788710af626174"}
Sep 16 18:32:07.237028 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:07.236678    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m" event={"ID":"10d1faf7-a952-473d-bb0c-f5b3c16b53fa","Type":"ContainerStarted","Data":"1c4c0f243350d99ffee52904f9b5bfb3d0722dc6b503d0619c50ce64df1d36d8"}
Sep 16 18:32:07.281471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:07.281388    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m" podStartSLOduration=1.0809444400000001 podStartE2EDuration="1.281373031s" totalImagesPullingTime="200.428591ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:32:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:32:07.274083585 +0000 UTC m=+950.830376896" watchObservedRunningTime="2026-09-16 18:32:07.281373031 +0000 UTC m=+950.837666344"
Sep 16 18:32:16.473259 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:16.473224    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:16.473656 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:16.473312    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:16.485540 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:16.485519    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:32:17.301135 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:32:17.301103    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m"
Sep 16 18:34:20.062855 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.062779    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7885-7ee711"]
Sep 16 18:34:20.066014 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.065996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:20.069814 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.069781    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fd5lk\""
Sep 16 18:34:20.081358 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.081335    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7885-7ee711"]
Sep 16 18:34:20.173974 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.173942    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"]
Sep 16 18:34:20.177478 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.177463    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:20.180904 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.180884    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-4ab744\""
Sep 16 18:34:20.181026 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.180946    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-4ab744-dockercfg-q265d\""
Sep 16 18:34:20.184186 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.184158    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkfqt\" (UniqueName: \"kubernetes.io/projected/6a96b227-f587-48e0-a220-535c9e2a86b5-kube-api-access-lkfqt\") pod \"test-curl-7885-7ee711\" (UID: \"6a96b227-f587-48e0-a220-535c9e2a86b5\") " pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:20.194966 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.194945    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"]
Sep 16 18:34:20.284973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.284945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkfqt\" (UniqueName: \"kubernetes.io/projected/6a96b227-f587-48e0-a220-535c9e2a86b5-kube-api-access-lkfqt\") pod \"test-curl-7885-7ee711\" (UID: \"6a96b227-f587-48e0-a220-535c9e2a86b5\") " pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:20.285110 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.284991    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ghvz\" (UniqueName: \"kubernetes.io/projected/55b96ade-9967-419b-9944-337be017cfc3-kube-api-access-8ghvz\") pod \"payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c\" (UID: \"55b96ade-9967-419b-9944-337be017cfc3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:20.285110 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.285031    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b96ade-9967-419b-9944-337be017cfc3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c\" (UID: \"55b96ade-9967-419b-9944-337be017cfc3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:20.301892 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.301872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkfqt\" (UniqueName: \"kubernetes.io/projected/6a96b227-f587-48e0-a220-535c9e2a86b5-kube-api-access-lkfqt\") pod \"test-curl-7885-7ee711\" (UID: \"6a96b227-f587-48e0-a220-535c9e2a86b5\") " pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:20.378077 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.378020    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:20.385496 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.385467    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ghvz\" (UniqueName: \"kubernetes.io/projected/55b96ade-9967-419b-9944-337be017cfc3-kube-api-access-8ghvz\") pod \"payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c\" (UID: \"55b96ade-9967-419b-9944-337be017cfc3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:20.385614 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.385517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b96ade-9967-419b-9944-337be017cfc3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c\" (UID: \"55b96ade-9967-419b-9944-337be017cfc3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:20.386203 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.386186    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"]
Sep 16 18:34:20.386203 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.386193    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b96ade-9967-419b-9944-337be017cfc3-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c\" (UID: \"55b96ade-9967-419b-9944-337be017cfc3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:20.390971 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.390953    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:20.414763 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.414715    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"]
Sep 16 18:34:20.420799 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.420777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ghvz\" (UniqueName: \"kubernetes.io/projected/55b96ade-9967-419b-9944-337be017cfc3-kube-api-access-8ghvz\") pod \"payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c\" (UID: \"55b96ade-9967-419b-9944-337be017cfc3\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:20.486454 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.486421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zh6g\" (UniqueName: \"kubernetes.io/projected/bdb7db43-f916-4c9a-bfe4-894a4deb2047-kube-api-access-5zh6g\") pod \"payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww\" (UID: \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:20.486454 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.486456    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb7db43-f916-4c9a-bfe4-894a4deb2047-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww\" (UID: \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:20.487249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.487226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:20.587394 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.587366    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zh6g\" (UniqueName: \"kubernetes.io/projected/bdb7db43-f916-4c9a-bfe4-894a4deb2047-kube-api-access-5zh6g\") pod \"payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww\" (UID: \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:20.587533 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.587400    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb7db43-f916-4c9a-bfe4-894a4deb2047-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww\" (UID: \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:20.588037 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.588020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb7db43-f916-4c9a-bfe4-894a4deb2047-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww\" (UID: \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:20.617822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.617796    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zh6g\" (UniqueName: \"kubernetes.io/projected/bdb7db43-f916-4c9a-bfe4-894a4deb2047-kube-api-access-5zh6g\") pod \"payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww\" (UID: \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\") " pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:20.692074 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.692051    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7885-7ee711"]
Sep 16 18:34:20.692176 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:34:20.692080    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6a96b227_f587_48e0_a220_535c9e2a86b5.slice/crio-f1baec7aad111b31eda46544737c5c3588513b781b78aabb5fa9410e41373310 WatchSource:0}: Error finding container f1baec7aad111b31eda46544737c5c3588513b781b78aabb5fa9410e41373310: Status 404 returned error can't find the container with id f1baec7aad111b31eda46544737c5c3588513b781b78aabb5fa9410e41373310
Sep 16 18:34:20.693928 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.693914    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:34:20.718714 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.717079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:20.799553 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.799524    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"]
Sep 16 18:34:20.799729 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:34:20.799708    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55b96ade_9967_419b_9944_337be017cfc3.slice/crio-5c6bf831132fcf573ad5a578dd8d38c0753979b05df4c2a0f9f262639721dc48 WatchSource:0}: Error finding container 5c6bf831132fcf573ad5a578dd8d38c0753979b05df4c2a0f9f262639721dc48: Status 404 returned error can't find the container with id 5c6bf831132fcf573ad5a578dd8d38c0753979b05df4c2a0f9f262639721dc48
Sep 16 18:34:20.876267 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.876232    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c" event={"ID":"55b96ade-9967-419b-9944-337be017cfc3","Type":"ContainerStarted","Data":"537e08d5a28378a4a0668e7e4f59783d689a338c544e329f2b9968050fb503c3"}
Sep 16 18:34:20.876412 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.876293    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c" event={"ID":"55b96ade-9967-419b-9944-337be017cfc3","Type":"ContainerStarted","Data":"5c6bf831132fcf573ad5a578dd8d38c0753979b05df4c2a0f9f262639721dc48"}
Sep 16 18:34:20.876412 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.876319    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:20.877570 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.877541    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7885-7ee711" event={"ID":"6a96b227-f587-48e0-a220-535c9e2a86b5","Type":"ContainerStarted","Data":"f1baec7aad111b31eda46544737c5c3588513b781b78aabb5fa9410e41373310"}
Sep 16 18:34:20.933366 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:20.933322    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c" podStartSLOduration=0.933304783 podStartE2EDuration="933.304783ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:20.932753491 +0000 UTC m=+1084.489046802" watchObservedRunningTime="2026-09-16 18:34:20.933304783 +0000 UTC m=+1084.489598093"
Sep 16 18:34:21.039431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:21.039251    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"]
Sep 16 18:34:21.041702 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:34:21.041678    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb7db43_f916_4c9a_bfe4_894a4deb2047.slice/crio-49f7af5329acf63bddac2033af7ed042e2922825579c7e431b11b255cb47940b WatchSource:0}: Error finding container 49f7af5329acf63bddac2033af7ed042e2922825579c7e431b11b255cb47940b: Status 404 returned error can't find the container with id 49f7af5329acf63bddac2033af7ed042e2922825579c7e431b11b255cb47940b
Sep 16 18:34:21.886256 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:21.886198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww" event={"ID":"bdb7db43-f916-4c9a-bfe4-894a4deb2047","Type":"ContainerStarted","Data":"ca68de78d1af91e8654c0aa749fffbd2abf3339f7d02a02a6911125a275d1784"}
Sep 16 18:34:21.886679 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:21.886274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww" event={"ID":"bdb7db43-f916-4c9a-bfe4-894a4deb2047","Type":"ContainerStarted","Data":"49f7af5329acf63bddac2033af7ed042e2922825579c7e431b11b255cb47940b"}
Sep 16 18:34:21.886679 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:21.886409    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:22.067277 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:22.067221    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww" podStartSLOduration=2.067187363 podStartE2EDuration="2.067187363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:22.055240417 +0000 UTC m=+1085.611533730" watchObservedRunningTime="2026-09-16 18:34:22.067187363 +0000 UTC m=+1085.623480676"
Sep 16 18:34:22.893888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:22.893857    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7885-7ee711" event={"ID":"6a96b227-f587-48e0-a220-535c9e2a86b5","Type":"ContainerStarted","Data":"c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4"}
Sep 16 18:34:22.978054 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:22.977997    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7885-7ee711" podStartSLOduration=0.912581961 podStartE2EDuration="2.977981776s" totalImagesPullingTime="2.065399815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:22.975941473 +0000 UTC m=+1086.532234785" watchObservedRunningTime="2026-09-16 18:34:22.977981776 +0000 UTC m=+1086.534275116"
Sep 16 18:34:23.439557 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.439528    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"]
Sep 16 18:34:23.443101 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.443083    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:23.451198 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.451177    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-909a40-dockercfg-687fs\""
Sep 16 18:34:23.451878 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.451861    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-909a40\""
Sep 16 18:34:23.498715 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.498648    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"]
Sep 16 18:34:23.517270 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.517228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b39d210-6738-4cf7-82e5-0f8025c69759-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl\" (UID: \"9b39d210-6738-4cf7-82e5-0f8025c69759\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:23.517417 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.517399    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtqlt\" (UniqueName: \"kubernetes.io/projected/9b39d210-6738-4cf7-82e5-0f8025c69759-kube-api-access-jtqlt\") pod \"payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl\" (UID: \"9b39d210-6738-4cf7-82e5-0f8025c69759\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:23.618591 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.618564    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b39d210-6738-4cf7-82e5-0f8025c69759-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl\" (UID: \"9b39d210-6738-4cf7-82e5-0f8025c69759\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:23.618751 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.618645    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtqlt\" (UniqueName: \"kubernetes.io/projected/9b39d210-6738-4cf7-82e5-0f8025c69759-kube-api-access-jtqlt\") pod \"payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl\" (UID: \"9b39d210-6738-4cf7-82e5-0f8025c69759\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:23.619335 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.619312    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b39d210-6738-4cf7-82e5-0f8025c69759-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl\" (UID: \"9b39d210-6738-4cf7-82e5-0f8025c69759\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:23.668730 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.668708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtqlt\" (UniqueName: \"kubernetes.io/projected/9b39d210-6738-4cf7-82e5-0f8025c69759-kube-api-access-jtqlt\") pod \"payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl\" (UID: \"9b39d210-6738-4cf7-82e5-0f8025c69759\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:23.753124 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:23.753055    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:24.056034 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:24.056009    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"]
Sep 16 18:34:24.058399 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:34:24.058370    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b39d210_6738_4cf7_82e5_0f8025c69759.slice/crio-6576be4febff2908004ffd25a4a5c2505716757839d9a6454f0a91b91c301c8c WatchSource:0}: Error finding container 6576be4febff2908004ffd25a4a5c2505716757839d9a6454f0a91b91c301c8c: Status 404 returned error can't find the container with id 6576be4febff2908004ffd25a4a5c2505716757839d9a6454f0a91b91c301c8c
Sep 16 18:34:24.412543 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:24.412503    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7885-7ee711"]
Sep 16 18:34:24.508611 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:24.508578    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7885-7ee711"]
Sep 16 18:34:24.905529 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:24.905496    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl" event={"ID":"9b39d210-6738-4cf7-82e5-0f8025c69759","Type":"ContainerStarted","Data":"98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b"}
Sep 16 18:34:24.905690 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:24.905556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl" event={"ID":"9b39d210-6738-4cf7-82e5-0f8025c69759","Type":"ContainerStarted","Data":"6576be4febff2908004ffd25a4a5c2505716757839d9a6454f0a91b91c301c8c"}
Sep 16 18:34:24.905690 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:24.905589    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:24.905795 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:24.905696    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7885-7ee711" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" containerName="test-curl-7885-7ee711" containerID="cri-o://c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4" gracePeriod=2
Sep 16 18:34:24.999414 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:24.999368    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl" podStartSLOduration=1.999351936 podStartE2EDuration="1.999351936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:24.998324909 +0000 UTC m=+1088.554618220" watchObservedRunningTime="2026-09-16 18:34:24.999351936 +0000 UTC m=+1088.555645247"
Sep 16 18:34:27.105673 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.105646    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:27.111703 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:34:27.111678    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7885-7ee711\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:27.149036 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.149014    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkfqt\" (UniqueName: \"kubernetes.io/projected/6a96b227-f587-48e0-a220-535c9e2a86b5-kube-api-access-lkfqt\") pod \"6a96b227-f587-48e0-a220-535c9e2a86b5\" (UID: \"6a96b227-f587-48e0-a220-535c9e2a86b5\") "
Sep 16 18:34:27.151049 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.151019    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a96b227-f587-48e0-a220-535c9e2a86b5-kube-api-access-lkfqt" (OuterVolumeSpecName: "kube-api-access-lkfqt") pod "6a96b227-f587-48e0-a220-535c9e2a86b5" (UID: "6a96b227-f587-48e0-a220-535c9e2a86b5"). InnerVolumeSpecName "kube-api-access-lkfqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:34:27.250775 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.250699    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkfqt\" (UniqueName: \"kubernetes.io/projected/6a96b227-f587-48e0-a220-535c9e2a86b5-kube-api-access-lkfqt\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:34:27.299657 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.299631    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"]
Sep 16 18:34:27.300057 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.300043    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" containerName="test-curl-7885-7ee711"
Sep 16 18:34:27.300119 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.300058    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" containerName="test-curl-7885-7ee711"
Sep 16 18:34:27.300173 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.300126    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" containerName="test-curl-7885-7ee711"
Sep 16 18:34:27.303241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.303203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:27.313839 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:34:27.313813    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7885-7ee711\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:27.319519 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.319497    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-d8a600\""
Sep 16 18:34:27.320072 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.320056    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-d8a600-dockercfg-v5ldr\""
Sep 16 18:34:27.341494 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.341468    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"]
Sep 16 18:34:27.351946 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.351926    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx8t6\" (UniqueName: \"kubernetes.io/projected/6cb57348-cfbe-4f66-af79-7511d3eca1e2-kube-api-access-tx8t6\") pod \"payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln\" (UID: \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:27.352042 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.351966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb57348-cfbe-4f66-af79-7511d3eca1e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln\" (UID: \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:27.452990 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.452962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx8t6\" (UniqueName: \"kubernetes.io/projected/6cb57348-cfbe-4f66-af79-7511d3eca1e2-kube-api-access-tx8t6\") pod \"payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln\" (UID: \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:27.453123 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.453027    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb57348-cfbe-4f66-af79-7511d3eca1e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln\" (UID: \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:27.453949 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.453924    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb57348-cfbe-4f66-af79-7511d3eca1e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln\" (UID: \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:27.469717 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.469693    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"]
Sep 16 18:34:27.473263 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.473249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:27.484815 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:34:27.484790    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7885-7ee711\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:27.503448 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.503399    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"]
Sep 16 18:34:27.506218 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.506187    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx8t6\" (UniqueName: \"kubernetes.io/projected/6cb57348-cfbe-4f66-af79-7511d3eca1e2-kube-api-access-tx8t6\") pod \"payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln\" (UID: \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:27.553682 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.553655    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/613f3e0a-d507-45e5-8f3d-2237044c7ffd-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf\" (UID: \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:27.553804 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.553692    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kx46\" (UniqueName: \"kubernetes.io/projected/613f3e0a-d507-45e5-8f3d-2237044c7ffd-kube-api-access-7kx46\") pod \"payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf\" (UID: \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:27.612899 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.612876    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:27.654588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.654555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/613f3e0a-d507-45e5-8f3d-2237044c7ffd-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf\" (UID: \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:27.654730 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.654605    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kx46\" (UniqueName: \"kubernetes.io/projected/613f3e0a-d507-45e5-8f3d-2237044c7ffd-kube-api-access-7kx46\") pod \"payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf\" (UID: \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:27.655337 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.655318    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/613f3e0a-d507-45e5-8f3d-2237044c7ffd-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf\" (UID: \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:27.699246 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.699220    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kx46\" (UniqueName: \"kubernetes.io/projected/613f3e0a-d507-45e5-8f3d-2237044c7ffd-kube-api-access-7kx46\") pod \"payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf\" (UID: \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:27.784041 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.783974    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:27.924743 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.924711    2576 generic.go:356] "Generic (PLEG): container finished" podID="6a96b227-f587-48e0-a220-535c9e2a86b5" containerID="c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4" exitCode=137
Sep 16 18:34:27.924891 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.924754    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:27.924891 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.924808    2576 scope.go:117] "RemoveContainer" containerID="c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4"
Sep 16 18:34:27.933725 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.933708    2576 scope.go:117] "RemoveContainer" containerID="c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4"
Sep 16 18:34:27.934050 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:34:27.934030    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4\": container with ID starting with c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4 not found: ID does not exist" containerID="c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4"
Sep 16 18:34:27.934123 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.934058    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4"} err="failed to get container status \"c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4\": rpc error: code = NotFound desc = could not find container \"c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4\": container with ID starting with c2e94f0332b17cda05b364d8312eb92a504b6b232cce585f06aac3510f1afda4 not found: ID does not exist"
Sep 16 18:34:27.956227 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:34:27.956182    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb57348_cfbe_4f66_af79_7511d3eca1e2.slice/crio-b63274bbeb1186649702401d99716f02570561582a90a509d580a34350c72714 WatchSource:0}: Error finding container b63274bbeb1186649702401d99716f02570561582a90a509d580a34350c72714: Status 404 returned error can't find the container with id b63274bbeb1186649702401d99716f02570561582a90a509d580a34350c72714
Sep 16 18:34:27.963938 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:27.963916    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"]
Sep 16 18:34:27.964766 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:34:27.964742    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7885-7ee711\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:27.966656 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:34:27.966635    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7885-7ee711\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:28.234638 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:28.234617    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"]
Sep 16 18:34:28.236589 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:34:28.236561    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613f3e0a_d507_45e5_8f3d_2237044c7ffd.slice/crio-82bc11d6fbc5615a1b3d21a1db05600e0cca801c47ed93dd1664a5c99d6ab5d1 WatchSource:0}: Error finding container 82bc11d6fbc5615a1b3d21a1db05600e0cca801c47ed93dd1664a5c99d6ab5d1: Status 404 returned error can't find the container with id 82bc11d6fbc5615a1b3d21a1db05600e0cca801c47ed93dd1664a5c99d6ab5d1
Sep 16 18:34:28.933285 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:28.933246    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf" event={"ID":"613f3e0a-d507-45e5-8f3d-2237044c7ffd","Type":"ContainerStarted","Data":"b59eeb367e60588a692e37a56e6071e1e7a333e426ef2e9f3f1c20d132923c56"}
Sep 16 18:34:28.933469 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:28.933331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf" event={"ID":"613f3e0a-d507-45e5-8f3d-2237044c7ffd","Type":"ContainerStarted","Data":"82bc11d6fbc5615a1b3d21a1db05600e0cca801c47ed93dd1664a5c99d6ab5d1"}
Sep 16 18:34:28.933469 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:28.933365    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:28.934982 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:28.934956    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln" event={"ID":"6cb57348-cfbe-4f66-af79-7511d3eca1e2","Type":"ContainerStarted","Data":"77a8bb1b73c293fe6eb9ccc9ff7f064a8dedfee7780ccb52f263a672113c66a5"}
Sep 16 18:34:28.935117 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:28.935001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln" event={"ID":"6cb57348-cfbe-4f66-af79-7511d3eca1e2","Type":"ContainerStarted","Data":"b63274bbeb1186649702401d99716f02570561582a90a509d580a34350c72714"}
Sep 16 18:34:28.935117 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:28.935026    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:28.944762 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:28.944737    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" path="/var/lib/kubelet/pods/6a96b227-f587-48e0-a220-535c9e2a86b5/volumes"
Sep 16 18:34:28.946324 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:34:28.946294    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7885-7ee711\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="6a96b227-f587-48e0-a220-535c9e2a86b5" pod="opendatahub/test-curl-7885-7ee711"
Sep 16 18:34:29.250566 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:29.250463    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf" podStartSLOduration=2.250446343 podStartE2EDuration="2.250446343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:29.216188676 +0000 UTC m=+1092.772481987" watchObservedRunningTime="2026-09-16 18:34:29.250446343 +0000 UTC m=+1092.806739653"
Sep 16 18:34:29.353831 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:29.353782    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln" podStartSLOduration=2.353767354 podStartE2EDuration="2.353767354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:29.350490193 +0000 UTC m=+1092.906783508" watchObservedRunningTime="2026-09-16 18:34:29.353767354 +0000 UTC m=+1092.910060724"
Sep 16 18:34:31.888405 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:31.888370    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:34:32.895876 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:32.895852    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:34:35.913382 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:35.913346    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:34:39.943881 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:39.943852    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:34:39.944263 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:39.943916    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:34:48.986606 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:48.986573    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"]
Sep 16 18:34:48.990997 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:48.990969    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:34:48.993610 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:48.993584    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-8b875d\""
Sep 16 18:34:49.057307 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.057281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hn5l\" (UniqueName: \"kubernetes.io/projected/5ca38593-2150-4c57-aa4b-4b1d957aba22-kube-api-access-4hn5l\") pod \"payload-processing-e2e-autores-8b875d-949df588f-m2cmk\" (UID: \"5ca38593-2150-4c57-aa4b-4b1d957aba22\") " pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:34:49.057463 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.057337    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ca38593-2150-4c57-aa4b-4b1d957aba22-plugins-config-volume\") pod \"payload-processing-e2e-autores-8b875d-949df588f-m2cmk\" (UID: \"5ca38593-2150-4c57-aa4b-4b1d957aba22\") " pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:34:49.158690 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.158664    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hn5l\" (UniqueName: \"kubernetes.io/projected/5ca38593-2150-4c57-aa4b-4b1d957aba22-kube-api-access-4hn5l\") pod \"payload-processing-e2e-autores-8b875d-949df588f-m2cmk\" (UID: \"5ca38593-2150-4c57-aa4b-4b1d957aba22\") " pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:34:49.158806 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.158718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ca38593-2150-4c57-aa4b-4b1d957aba22-plugins-config-volume\") pod \"payload-processing-e2e-autores-8b875d-949df588f-m2cmk\" (UID: \"5ca38593-2150-4c57-aa4b-4b1d957aba22\") " pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:34:49.159367 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.159350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ca38593-2150-4c57-aa4b-4b1d957aba22-plugins-config-volume\") pod \"payload-processing-e2e-autores-8b875d-949df588f-m2cmk\" (UID: \"5ca38593-2150-4c57-aa4b-4b1d957aba22\") " pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:34:49.196597 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.196574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hn5l\" (UniqueName: \"kubernetes.io/projected/5ca38593-2150-4c57-aa4b-4b1d957aba22-kube-api-access-4hn5l\") pod \"payload-processing-e2e-autores-8b875d-949df588f-m2cmk\" (UID: \"5ca38593-2150-4c57-aa4b-4b1d957aba22\") " pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:34:49.223286 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.223260    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"]
Sep 16 18:34:49.235762 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.235739    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"]
Sep 16 18:34:49.239702 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.239656    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:34:49.246311 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.246289    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-8b875d-dockercfg-cbjvj\""
Sep 16 18:34:49.259443 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.259421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf\" (UID: \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:34:49.259538 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.259487    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db58t\" (UniqueName: \"kubernetes.io/projected/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-kube-api-access-db58t\") pod \"payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf\" (UID: \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:34:49.302664 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.302644    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:34:49.345993 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.345960    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"]
Sep 16 18:34:49.360149 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.360116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf\" (UID: \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:34:49.360302 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.360254    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db58t\" (UniqueName: \"kubernetes.io/projected/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-kube-api-access-db58t\") pod \"payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf\" (UID: \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:34:49.361482 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.361441    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf\" (UID: \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:34:49.390051 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.390019    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-db58t\" (UniqueName: \"kubernetes.io/projected/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-kube-api-access-db58t\") pod \"payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf\" (UID: \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:34:49.549783 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.549712    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:34:49.648475 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.648441    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"]
Sep 16 18:34:49.651355 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:34:49.651327    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca38593_2150_4c57_aa4b_4b1d957aba22.slice/crio-c4931d2f1d184fb0e0a17c35a652bbae7a2fc112aa489346d0b83c5e6b9ea882 WatchSource:0}: Error finding container c4931d2f1d184fb0e0a17c35a652bbae7a2fc112aa489346d0b83c5e6b9ea882: Status 404 returned error can't find the container with id c4931d2f1d184fb0e0a17c35a652bbae7a2fc112aa489346d0b83c5e6b9ea882
Sep 16 18:34:49.903417 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:49.903394    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"]
Sep 16 18:34:49.905755 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:34:49.905728    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf40f45c9_42e2_4c92_b6c5_a0eb41590ef4.slice/crio-aaa4e38739c7a2a0183e6fbb4327d91ac6750cf9f15e6b53a4f1ca91a303b6d7 WatchSource:0}: Error finding container aaa4e38739c7a2a0183e6fbb4327d91ac6750cf9f15e6b53a4f1ca91a303b6d7: Status 404 returned error can't find the container with id aaa4e38739c7a2a0183e6fbb4327d91ac6750cf9f15e6b53a4f1ca91a303b6d7
Sep 16 18:34:50.049343 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:50.049302    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf" event={"ID":"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4","Type":"ContainerStarted","Data":"6fa62ecf1a0ece9528b253f8ed5a28aad87b7830d9c9d4ebb524336d5fd6bee4"}
Sep 16 18:34:50.049804 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:50.049367    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf" event={"ID":"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4","Type":"ContainerStarted","Data":"aaa4e38739c7a2a0183e6fbb4327d91ac6750cf9f15e6b53a4f1ca91a303b6d7"}
Sep 16 18:34:50.049804 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:50.049416    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:34:50.051251 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:50.051197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk" event={"ID":"5ca38593-2150-4c57-aa4b-4b1d957aba22","Type":"ContainerStarted","Data":"d4d1ff0f341ce9915b1d4721895e6d86bc9d3e241be92ee05439e91698dfbd8d"}
Sep 16 18:34:50.051390 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:50.051269    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk" event={"ID":"5ca38593-2150-4c57-aa4b-4b1d957aba22","Type":"ContainerStarted","Data":"c4931d2f1d184fb0e0a17c35a652bbae7a2fc112aa489346d0b83c5e6b9ea882"}
Sep 16 18:34:50.051390 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:50.051299    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:34:50.105277 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:50.105228    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf" podStartSLOduration=1.105196127 podStartE2EDuration="1.105196127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:50.103672619 +0000 UTC m=+1113.659965935" watchObservedRunningTime="2026-09-16 18:34:50.105196127 +0000 UTC m=+1113.661489437"
Sep 16 18:34:50.138624 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:50.138578    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk" podStartSLOduration=2.138559145 podStartE2EDuration="2.138559145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:50.137825868 +0000 UTC m=+1113.694119191" watchObservedRunningTime="2026-09-16 18:34:50.138559145 +0000 UTC m=+1113.694852457"
Sep 16 18:34:57.964945 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:57.964911    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"]
Sep 16 18:34:57.969689 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:57.969673    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:34:57.972186 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:57.972163    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-119aafd2\""
Sep 16 18:34:57.972298 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:57.972163    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-119aafd2-dockercfg-hppvb\""
Sep 16 18:34:58.003690 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:58.003663    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"]
Sep 16 18:34:58.036817 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:58.036796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2frhz\" (UniqueName: \"kubernetes.io/projected/ed781521-e953-4135-8031-b12c9c921a8c-kube-api-access-2frhz\") pod \"payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v\" (UID: \"ed781521-e953-4135-8031-b12c9c921a8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:34:58.036916 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:58.036893    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed781521-e953-4135-8031-b12c9c921a8c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v\" (UID: \"ed781521-e953-4135-8031-b12c9c921a8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:34:58.138190 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:58.138157    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed781521-e953-4135-8031-b12c9c921a8c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v\" (UID: \"ed781521-e953-4135-8031-b12c9c921a8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:34:58.138365 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:58.138272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2frhz\" (UniqueName: \"kubernetes.io/projected/ed781521-e953-4135-8031-b12c9c921a8c-kube-api-access-2frhz\") pod \"payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v\" (UID: \"ed781521-e953-4135-8031-b12c9c921a8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:34:58.138974 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:58.138955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed781521-e953-4135-8031-b12c9c921a8c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v\" (UID: \"ed781521-e953-4135-8031-b12c9c921a8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:34:58.160007 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:58.159983    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2frhz\" (UniqueName: \"kubernetes.io/projected/ed781521-e953-4135-8031-b12c9c921a8c-kube-api-access-2frhz\") pod \"payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v\" (UID: \"ed781521-e953-4135-8031-b12c9c921a8c\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:34:58.280498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:58.280435    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:34:58.570667 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:58.570646    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"]
Sep 16 18:34:58.573464 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:34:58.573433    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded781521_e953_4135_8031_b12c9c921a8c.slice/crio-5afff24fc2b542076fec3b0872c659d0311169e4c8287a1ac5374d641c0ccec2 WatchSource:0}: Error finding container 5afff24fc2b542076fec3b0872c659d0311169e4c8287a1ac5374d641c0ccec2: Status 404 returned error can't find the container with id 5afff24fc2b542076fec3b0872c659d0311169e4c8287a1ac5374d641c0ccec2
Sep 16 18:34:59.096912 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:59.096879    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v" event={"ID":"ed781521-e953-4135-8031-b12c9c921a8c","Type":"ContainerStarted","Data":"2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f"}
Sep 16 18:34:59.097308 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:59.096931    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v" event={"ID":"ed781521-e953-4135-8031-b12c9c921a8c","Type":"ContainerStarted","Data":"5afff24fc2b542076fec3b0872c659d0311169e4c8287a1ac5374d641c0ccec2"}
Sep 16 18:34:59.097308 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:59.096976    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:34:59.162308 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:34:59.162258    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v" podStartSLOduration=2.162243546 podStartE2EDuration="2.162243546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:34:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:34:59.161226244 +0000 UTC m=+1122.717519550" watchObservedRunningTime="2026-09-16 18:34:59.162243546 +0000 UTC m=+1122.718536858"
Sep 16 18:35:01.057799 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:01.057716    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:35:01.058148 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:01.057892    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:35:07.393179 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.393135    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"]
Sep 16 18:35:07.396954 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.396938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:07.399874 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.399851    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-6ef84e\""
Sep 16 18:35:07.401247 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.401230    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-6ef84e-dockercfg-brwkq\""
Sep 16 18:35:07.413166 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.413145    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"]
Sep 16 18:35:07.416694 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.416680    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:07.439821 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.439801    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"]
Sep 16 18:35:07.451145 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.451111    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"]
Sep 16 18:35:07.525848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.525820    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88590f47-c61b-4e8b-b871-c479ccc641cf-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl\" (UID: \"88590f47-c61b-4e8b-b871-c479ccc641cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:07.525974 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.525851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj75m\" (UniqueName: \"kubernetes.io/projected/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-kube-api-access-tj75m\") pod \"payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm\" (UID: \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:07.525974 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.525871    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkjp4\" (UniqueName: \"kubernetes.io/projected/88590f47-c61b-4e8b-b871-c479ccc641cf-kube-api-access-pkjp4\") pod \"payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl\" (UID: \"88590f47-c61b-4e8b-b871-c479ccc641cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:07.526054 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.525982    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm\" (UID: \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:07.626650 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.626621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88590f47-c61b-4e8b-b871-c479ccc641cf-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl\" (UID: \"88590f47-c61b-4e8b-b871-c479ccc641cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:07.626810 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.626656    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj75m\" (UniqueName: \"kubernetes.io/projected/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-kube-api-access-tj75m\") pod \"payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm\" (UID: \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:07.626810 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.626677    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkjp4\" (UniqueName: \"kubernetes.io/projected/88590f47-c61b-4e8b-b871-c479ccc641cf-kube-api-access-pkjp4\") pod \"payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl\" (UID: \"88590f47-c61b-4e8b-b871-c479ccc641cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:07.626810 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.626734    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm\" (UID: \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:07.627342 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.627323    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88590f47-c61b-4e8b-b871-c479ccc641cf-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl\" (UID: \"88590f47-c61b-4e8b-b871-c479ccc641cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:07.627404 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.627373    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm\" (UID: \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:07.657756 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.657705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj75m\" (UniqueName: \"kubernetes.io/projected/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-kube-api-access-tj75m\") pod \"payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm\" (UID: \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:07.676221 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.676185    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkjp4\" (UniqueName: \"kubernetes.io/projected/88590f47-c61b-4e8b-b871-c479ccc641cf-kube-api-access-pkjp4\") pod \"payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl\" (UID: \"88590f47-c61b-4e8b-b871-c479ccc641cf\") " pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:07.707310 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.707286    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:07.726400 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:07.726375    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:08.014838 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:08.012752    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"]
Sep 16 18:35:08.022272 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:35:08.022246    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2f74f1_d75b_44fa_9ebb_12c0b566796b.slice/crio-d28778ecd65346f9036cec393df75d591461bcb2e207fbdb43dd4afad872b1d0 WatchSource:0}: Error finding container d28778ecd65346f9036cec393df75d591461bcb2e207fbdb43dd4afad872b1d0: Status 404 returned error can't find the container with id d28778ecd65346f9036cec393df75d591461bcb2e207fbdb43dd4afad872b1d0
Sep 16 18:35:08.044019 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:08.043998    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"]
Sep 16 18:35:08.045820 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:35:08.045797    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88590f47_c61b_4e8b_b871_c479ccc641cf.slice/crio-3884e7c7998c722e1b33f3802f2f215d965ed971a08b2e106f0b1953f8e5ba5f WatchSource:0}: Error finding container 3884e7c7998c722e1b33f3802f2f215d965ed971a08b2e106f0b1953f8e5ba5f: Status 404 returned error can't find the container with id 3884e7c7998c722e1b33f3802f2f215d965ed971a08b2e106f0b1953f8e5ba5f
Sep 16 18:35:08.144505 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:08.144469    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl" event={"ID":"88590f47-c61b-4e8b-b871-c479ccc641cf","Type":"ContainerStarted","Data":"05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027"}
Sep 16 18:35:08.144659 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:08.144533    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl" event={"ID":"88590f47-c61b-4e8b-b871-c479ccc641cf","Type":"ContainerStarted","Data":"3884e7c7998c722e1b33f3802f2f215d965ed971a08b2e106f0b1953f8e5ba5f"}
Sep 16 18:35:08.144659 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:08.144562    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:08.146002 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:08.145982    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm" event={"ID":"2e2f74f1-d75b-44fa-9ebb-12c0b566796b","Type":"ContainerStarted","Data":"e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958"}
Sep 16 18:35:08.146102 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:08.146020    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm" event={"ID":"2e2f74f1-d75b-44fa-9ebb-12c0b566796b","Type":"ContainerStarted","Data":"d28778ecd65346f9036cec393df75d591461bcb2e207fbdb43dd4afad872b1d0"}
Sep 16 18:35:08.146162 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:08.146136    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:08.209242 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:08.209182    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl" podStartSLOduration=1.209165569 podStartE2EDuration="1.209165569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:08.207864988 +0000 UTC m=+1131.764158299" watchObservedRunningTime="2026-09-16 18:35:08.209165569 +0000 UTC m=+1131.765458909"
Sep 16 18:35:10.104314 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:10.104280    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:35:10.230959 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:10.230908    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm" podStartSLOduration=3.230891594 podStartE2EDuration="3.230891594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:08.269933664 +0000 UTC m=+1131.826226976" watchObservedRunningTime="2026-09-16 18:35:10.230891594 +0000 UTC m=+1133.787184905"
Sep 16 18:35:19.154405 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:19.154372    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:35:19.154887 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:19.154698    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:35:33.240469 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:33.240433    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"]
Sep 16 18:35:33.240856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:33.240619    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v" podUID="ed781521-e953-4135-8031-b12c9c921a8c" containerName="payload-pre-processing" containerID="cri-o://2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f" gracePeriod=30
Sep 16 18:35:33.487392 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:33.487364    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:35:33.580682 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:33.580609    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed781521-e953-4135-8031-b12c9c921a8c-plugins-config-volume\") pod \"ed781521-e953-4135-8031-b12c9c921a8c\" (UID: \"ed781521-e953-4135-8031-b12c9c921a8c\") "
Sep 16 18:35:33.580807 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:33.580749    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2frhz\" (UniqueName: \"kubernetes.io/projected/ed781521-e953-4135-8031-b12c9c921a8c-kube-api-access-2frhz\") pod \"ed781521-e953-4135-8031-b12c9c921a8c\" (UID: \"ed781521-e953-4135-8031-b12c9c921a8c\") "
Sep 16 18:35:33.580939 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:33.580917    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed781521-e953-4135-8031-b12c9c921a8c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ed781521-e953-4135-8031-b12c9c921a8c" (UID: "ed781521-e953-4135-8031-b12c9c921a8c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:35:33.581060 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:33.581036    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ed781521-e953-4135-8031-b12c9c921a8c-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:35:33.582772 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:33.582746    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed781521-e953-4135-8031-b12c9c921a8c-kube-api-access-2frhz" (OuterVolumeSpecName: "kube-api-access-2frhz") pod "ed781521-e953-4135-8031-b12c9c921a8c" (UID: "ed781521-e953-4135-8031-b12c9c921a8c"). InnerVolumeSpecName "kube-api-access-2frhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:35:33.681587 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:33.681558    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2frhz\" (UniqueName: \"kubernetes.io/projected/ed781521-e953-4135-8031-b12c9c921a8c-kube-api-access-2frhz\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:35:34.310371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.310340    2576 generic.go:356] "Generic (PLEG): container finished" podID="ed781521-e953-4135-8031-b12c9c921a8c" containerID="2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f" exitCode=0
Sep 16 18:35:34.310771 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.310385    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"
Sep 16 18:35:34.310771 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.310425    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v" event={"ID":"ed781521-e953-4135-8031-b12c9c921a8c","Type":"ContainerDied","Data":"2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f"}
Sep 16 18:35:34.310771 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.310477    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v" event={"ID":"ed781521-e953-4135-8031-b12c9c921a8c","Type":"ContainerDied","Data":"5afff24fc2b542076fec3b0872c659d0311169e4c8287a1ac5374d641c0ccec2"}
Sep 16 18:35:34.310771 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.310501    2576 scope.go:117] "RemoveContainer" containerID="2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f"
Sep 16 18:35:34.319573 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.319558    2576 scope.go:117] "RemoveContainer" containerID="2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f"
Sep 16 18:35:34.319847 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:35:34.319831    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f\": container with ID starting with 2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f not found: ID does not exist" containerID="2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f"
Sep 16 18:35:34.319892 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.319856    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f"} err="failed to get container status \"2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f\": rpc error: code = NotFound desc = could not find container \"2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f\": container with ID starting with 2f985fc29427c3f1431c52f7e6c4f7b66848766cb46ff28b75653dff7611461f not found: ID does not exist"
Sep 16 18:35:34.348333 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.348308    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"]
Sep 16 18:35:34.350578 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.350557    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v"]
Sep 16 18:35:34.945068 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:34.945036    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed781521-e953-4135-8031-b12c9c921a8c" path="/var/lib/kubelet/pods/ed781521-e953-4135-8031-b12c9c921a8c/volumes"
Sep 16 18:35:49.059706 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.059635    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"]
Sep 16 18:35:49.060141 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.060106    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed781521-e953-4135-8031-b12c9c921a8c" containerName="payload-pre-processing"
Sep 16 18:35:49.060141 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.060118    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed781521-e953-4135-8031-b12c9c921a8c" containerName="payload-pre-processing"
Sep 16 18:35:49.060234 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.060201    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed781521-e953-4135-8031-b12c9c921a8c" containerName="payload-pre-processing"
Sep 16 18:35:49.063174 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.063152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:35:49.067721 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.067693    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-ad2bfa56-dockercfg-4km4m\""
Sep 16 18:35:49.067721 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.067704    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-ad2bfa56\""
Sep 16 18:35:49.090384 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.090361    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"]
Sep 16 18:35:49.121801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.121776    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zng8\" (UniqueName: \"kubernetes.io/projected/89625b73-61c7-4a6d-b246-14d76d5f61ad-kube-api-access-7zng8\") pod \"payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b\" (UID: \"89625b73-61c7-4a6d-b246-14d76d5f61ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:35:49.121927 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.121903    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89625b73-61c7-4a6d-b246-14d76d5f61ad-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b\" (UID: \"89625b73-61c7-4a6d-b246-14d76d5f61ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:35:49.223309 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.223279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89625b73-61c7-4a6d-b246-14d76d5f61ad-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b\" (UID: \"89625b73-61c7-4a6d-b246-14d76d5f61ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:35:49.223455 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.223394    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zng8\" (UniqueName: \"kubernetes.io/projected/89625b73-61c7-4a6d-b246-14d76d5f61ad-kube-api-access-7zng8\") pod \"payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b\" (UID: \"89625b73-61c7-4a6d-b246-14d76d5f61ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:35:49.223975 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.223959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89625b73-61c7-4a6d-b246-14d76d5f61ad-plugins-config-volume\") pod \"payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b\" (UID: \"89625b73-61c7-4a6d-b246-14d76d5f61ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:35:49.241756 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.241733    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zng8\" (UniqueName: \"kubernetes.io/projected/89625b73-61c7-4a6d-b246-14d76d5f61ad-kube-api-access-7zng8\") pod \"payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b\" (UID: \"89625b73-61c7-4a6d-b246-14d76d5f61ad\") " pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:35:49.373717 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.373644    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:35:49.688150 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:49.688119    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"]
Sep 16 18:35:49.699279 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:35:49.699246    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89625b73_61c7_4a6d_b246_14d76d5f61ad.slice/crio-afa4a3327615156ef1493432ca7523e1eb77cd4893d478967ca01cb170b4c2a1 WatchSource:0}: Error finding container afa4a3327615156ef1493432ca7523e1eb77cd4893d478967ca01cb170b4c2a1: Status 404 returned error can't find the container with id afa4a3327615156ef1493432ca7523e1eb77cd4893d478967ca01cb170b4c2a1
Sep 16 18:35:50.397001 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:50.396962    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b" event={"ID":"89625b73-61c7-4a6d-b246-14d76d5f61ad","Type":"ContainerStarted","Data":"612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4"}
Sep 16 18:35:50.397460 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:50.397120    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:35:50.397460 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:50.397165    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b" event={"ID":"89625b73-61c7-4a6d-b246-14d76d5f61ad","Type":"ContainerStarted","Data":"afa4a3327615156ef1493432ca7523e1eb77cd4893d478967ca01cb170b4c2a1"}
Sep 16 18:35:50.449283 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:35:50.449231    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b" podStartSLOduration=2.449200545 podStartE2EDuration="2.449200545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:35:50.448790773 +0000 UTC m=+1174.005084079" watchObservedRunningTime="2026-09-16 18:35:50.449200545 +0000 UTC m=+1174.005493856"
Sep 16 18:36:01.404366 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:01.404336    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:36:13.915363 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:13.915326    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"]
Sep 16 18:36:13.926187 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:13.926155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"
Sep 16 18:36:13.934482 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:13.934457    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgwrl\""
Sep 16 18:36:13.934801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:13.934775    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"]
Sep 16 18:36:14.048336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:14.048300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e092a3e-92a7-426a-8526-e2701c5127c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42\" (UID: \"2e092a3e-92a7-426a-8526-e2701c5127c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"
Sep 16 18:36:14.149178 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:14.149152    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e092a3e-92a7-426a-8526-e2701c5127c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42\" (UID: \"2e092a3e-92a7-426a-8526-e2701c5127c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"
Sep 16 18:36:14.149725 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:14.149707    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e092a3e-92a7-426a-8526-e2701c5127c0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42\" (UID: \"2e092a3e-92a7-426a-8526-e2701c5127c0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"
Sep 16 18:36:14.238072 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:14.238011    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"
Sep 16 18:36:14.537056 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:14.537034    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"]
Sep 16 18:36:14.539769 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:36:14.539744    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e092a3e_92a7_426a_8526_e2701c5127c0.slice/crio-339687b4793ab721b73805b08556ab8d9877259e58dae0a12bc88c66f4272105 WatchSource:0}: Error finding container 339687b4793ab721b73805b08556ab8d9877259e58dae0a12bc88c66f4272105: Status 404 returned error can't find the container with id 339687b4793ab721b73805b08556ab8d9877259e58dae0a12bc88c66f4272105
Sep 16 18:36:15.539150 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:15.539106    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42" event={"ID":"2e092a3e-92a7-426a-8526-e2701c5127c0","Type":"ContainerStarted","Data":"339687b4793ab721b73805b08556ab8d9877259e58dae0a12bc88c66f4272105"}
Sep 16 18:36:17.554319 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:17.554286    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42" event={"ID":"2e092a3e-92a7-426a-8526-e2701c5127c0","Type":"ContainerStarted","Data":"1fe60f0f4f0679cfc63f70b23a895ba2a4de12e7845592fd6b6f163e531949b0"}
Sep 16 18:36:17.577470 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:17.577423    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42" podStartSLOduration=2.317788902 podStartE2EDuration="4.577406601s" totalImagesPullingTime="2.259617699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:36:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:36:17.57668039 +0000 UTC m=+1201.132973706" watchObservedRunningTime="2026-09-16 18:36:17.577406601 +0000 UTC m=+1201.133699964"
Sep 16 18:36:18.560856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:18.560819    2576 generic.go:356] "Generic (PLEG): container finished" podID="2e092a3e-92a7-426a-8526-e2701c5127c0" containerID="1fe60f0f4f0679cfc63f70b23a895ba2a4de12e7845592fd6b6f163e531949b0" exitCode=0
Sep 16 18:36:18.561230 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:18.560905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42" event={"ID":"2e092a3e-92a7-426a-8526-e2701c5127c0","Type":"ContainerDied","Data":"1fe60f0f4f0679cfc63f70b23a895ba2a4de12e7845592fd6b6f163e531949b0"}
Sep 16 18:36:19.706032 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:19.706006    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"
Sep 16 18:36:19.806905 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:19.806878    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e092a3e-92a7-426a-8526-e2701c5127c0-maas-api-service-ca\") pod \"2e092a3e-92a7-426a-8526-e2701c5127c0\" (UID: \"2e092a3e-92a7-426a-8526-e2701c5127c0\") "
Sep 16 18:36:19.807236 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:19.807194    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e092a3e-92a7-426a-8526-e2701c5127c0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2e092a3e-92a7-426a-8526-e2701c5127c0" (UID: "2e092a3e-92a7-426a-8526-e2701c5127c0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:36:19.908612 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:19.908583    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e092a3e-92a7-426a-8526-e2701c5127c0-maas-api-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:36:20.573864 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:20.573834    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"
Sep 16 18:36:20.574158 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:20.573866    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42" event={"ID":"2e092a3e-92a7-426a-8526-e2701c5127c0","Type":"ContainerDied","Data":"339687b4793ab721b73805b08556ab8d9877259e58dae0a12bc88c66f4272105"}
Sep 16 18:36:20.574158 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:20.573911    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="339687b4793ab721b73805b08556ab8d9877259e58dae0a12bc88c66f4272105"
Sep 16 18:36:25.208265 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.208235    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"]
Sep 16 18:36:25.208649 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.208430    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b" podUID="89625b73-61c7-4a6d-b246-14d76d5f61ad" containerName="payload-processing" containerID="cri-o://612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4" gracePeriod=30
Sep 16 18:36:25.457367 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.457340    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:36:25.560197 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.560122    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89625b73-61c7-4a6d-b246-14d76d5f61ad-plugins-config-volume\") pod \"89625b73-61c7-4a6d-b246-14d76d5f61ad\" (UID: \"89625b73-61c7-4a6d-b246-14d76d5f61ad\") "
Sep 16 18:36:25.560364 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.560271    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zng8\" (UniqueName: \"kubernetes.io/projected/89625b73-61c7-4a6d-b246-14d76d5f61ad-kube-api-access-7zng8\") pod \"89625b73-61c7-4a6d-b246-14d76d5f61ad\" (UID: \"89625b73-61c7-4a6d-b246-14d76d5f61ad\") "
Sep 16 18:36:25.560505 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.560482    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89625b73-61c7-4a6d-b246-14d76d5f61ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89625b73-61c7-4a6d-b246-14d76d5f61ad" (UID: "89625b73-61c7-4a6d-b246-14d76d5f61ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:36:25.562278 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.562247    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89625b73-61c7-4a6d-b246-14d76d5f61ad-kube-api-access-7zng8" (OuterVolumeSpecName: "kube-api-access-7zng8") pod "89625b73-61c7-4a6d-b246-14d76d5f61ad" (UID: "89625b73-61c7-4a6d-b246-14d76d5f61ad"). InnerVolumeSpecName "kube-api-access-7zng8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:36:25.601333 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.601298    2576 generic.go:356] "Generic (PLEG): container finished" podID="89625b73-61c7-4a6d-b246-14d76d5f61ad" containerID="612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4" exitCode=0
Sep 16 18:36:25.601470 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.601341    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"
Sep 16 18:36:25.601470 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.601377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b" event={"ID":"89625b73-61c7-4a6d-b246-14d76d5f61ad","Type":"ContainerDied","Data":"612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4"}
Sep 16 18:36:25.601470 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.601425    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b" event={"ID":"89625b73-61c7-4a6d-b246-14d76d5f61ad","Type":"ContainerDied","Data":"afa4a3327615156ef1493432ca7523e1eb77cd4893d478967ca01cb170b4c2a1"}
Sep 16 18:36:25.601470 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.601449    2576 scope.go:117] "RemoveContainer" containerID="612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4"
Sep 16 18:36:25.610801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.610739    2576 scope.go:117] "RemoveContainer" containerID="612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4"
Sep 16 18:36:25.611125 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:36:25.611065    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4\": container with ID starting with 612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4 not found: ID does not exist" containerID="612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4"
Sep 16 18:36:25.611125 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.611099    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4"} err="failed to get container status \"612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4\": rpc error: code = NotFound desc = could not find container \"612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4\": container with ID starting with 612af9883c93e1e0ff54f861744c29469004cd04e697ed786c3f4350be4b09c4 not found: ID does not exist"
Sep 16 18:36:25.661132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.661107    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zng8\" (UniqueName: \"kubernetes.io/projected/89625b73-61c7-4a6d-b246-14d76d5f61ad-kube-api-access-7zng8\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:36:25.661132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.661132    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89625b73-61c7-4a6d-b246-14d76d5f61ad-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:36:25.704499 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.704470    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"]
Sep 16 18:36:25.718969 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.718942    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b"]
Sep 16 18:36:25.802456 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.802426    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"]
Sep 16 18:36:25.816253 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:25.816178    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-ad2bfa56-lkg42"]
Sep 16 18:36:26.945443 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:26.945403    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e092a3e-92a7-426a-8526-e2701c5127c0" path="/var/lib/kubelet/pods/2e092a3e-92a7-426a-8526-e2701c5127c0/volumes"
Sep 16 18:36:26.946012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:26.945942    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89625b73-61c7-4a6d-b246-14d76d5f61ad" path="/var/lib/kubelet/pods/89625b73-61c7-4a6d-b246-14d76d5f61ad/volumes"
Sep 16 18:36:39.020461 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.020419    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"]
Sep 16 18:36:39.021075 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.021055    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e092a3e-92a7-426a-8526-e2701c5127c0" containerName="revoke-keys"
Sep 16 18:36:39.021169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.021077    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e092a3e-92a7-426a-8526-e2701c5127c0" containerName="revoke-keys"
Sep 16 18:36:39.021169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.021118    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89625b73-61c7-4a6d-b246-14d76d5f61ad" containerName="payload-processing"
Sep 16 18:36:39.021169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.021129    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="89625b73-61c7-4a6d-b246-14d76d5f61ad" containerName="payload-processing"
Sep 16 18:36:39.021352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.021278    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="89625b73-61c7-4a6d-b246-14d76d5f61ad" containerName="payload-processing"
Sep 16 18:36:39.021352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.021294    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e092a3e-92a7-426a-8526-e2701c5127c0" containerName="revoke-keys"
Sep 16 18:36:39.025820 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.025799    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"
Sep 16 18:36:39.029056 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.029038    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgwrl\""
Sep 16 18:36:39.045155 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.045130    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"]
Sep 16 18:36:39.089694 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.089665    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6a05b3-a9bb-4c05-b00b-8665f6141000-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8b875d-66jkh\" (UID: \"5a6a05b3-a9bb-4c05-b00b-8665f6141000\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"
Sep 16 18:36:39.190532 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.190500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6a05b3-a9bb-4c05-b00b-8665f6141000-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8b875d-66jkh\" (UID: \"5a6a05b3-a9bb-4c05-b00b-8665f6141000\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"
Sep 16 18:36:39.191086 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.191067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6a05b3-a9bb-4c05-b00b-8665f6141000-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-8b875d-66jkh\" (UID: \"5a6a05b3-a9bb-4c05-b00b-8665f6141000\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"
Sep 16 18:36:39.336357 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.336289    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"
Sep 16 18:36:39.613106 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.613032    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"]
Sep 16 18:36:39.616668 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:36:39.616638    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6a05b3_a9bb_4c05_b00b_8665f6141000.slice/crio-87f0069ee221aaff5452279e225221e3ed140e197cc8ab15ed894ae37a234525 WatchSource:0}: Error finding container 87f0069ee221aaff5452279e225221e3ed140e197cc8ab15ed894ae37a234525: Status 404 returned error can't find the container with id 87f0069ee221aaff5452279e225221e3ed140e197cc8ab15ed894ae37a234525
Sep 16 18:36:39.684859 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:39.684830    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh" event={"ID":"5a6a05b3-a9bb-4c05-b00b-8665f6141000","Type":"ContainerStarted","Data":"87f0069ee221aaff5452279e225221e3ed140e197cc8ab15ed894ae37a234525"}
Sep 16 18:36:40.692562 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:40.692528    2576 generic.go:356] "Generic (PLEG): container finished" podID="5a6a05b3-a9bb-4c05-b00b-8665f6141000" containerID="351a3bf9327b23917625fa1020b4ed9949a1ff0d0f8f1d01d1f0425a1204f74d" exitCode=0
Sep 16 18:36:40.692944 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:40.692598    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh" event={"ID":"5a6a05b3-a9bb-4c05-b00b-8665f6141000","Type":"ContainerDied","Data":"351a3bf9327b23917625fa1020b4ed9949a1ff0d0f8f1d01d1f0425a1204f74d"}
Sep 16 18:36:41.839688 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:41.839664    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"
Sep 16 18:36:41.924605 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:41.924576    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6a05b3-a9bb-4c05-b00b-8665f6141000-maas-api-service-ca\") pod \"5a6a05b3-a9bb-4c05-b00b-8665f6141000\" (UID: \"5a6a05b3-a9bb-4c05-b00b-8665f6141000\") "
Sep 16 18:36:41.924927 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:41.924903    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a6a05b3-a9bb-4c05-b00b-8665f6141000-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5a6a05b3-a9bb-4c05-b00b-8665f6141000" (UID: "5a6a05b3-a9bb-4c05-b00b-8665f6141000"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:36:42.025991 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:42.025806    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a6a05b3-a9bb-4c05-b00b-8665f6141000-maas-api-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:36:42.705767 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:42.705738    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"
Sep 16 18:36:42.705767 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:42.705740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh" event={"ID":"5a6a05b3-a9bb-4c05-b00b-8665f6141000","Type":"ContainerDied","Data":"87f0069ee221aaff5452279e225221e3ed140e197cc8ab15ed894ae37a234525"}
Sep 16 18:36:42.705966 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:42.705795    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87f0069ee221aaff5452279e225221e3ed140e197cc8ab15ed894ae37a234525"
Sep 16 18:36:47.594798 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.594758    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"]
Sep 16 18:36:47.595469 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.595450    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a6a05b3-a9bb-4c05-b00b-8665f6141000" containerName="revoke-keys"
Sep 16 18:36:47.595525 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.595472    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a6a05b3-a9bb-4c05-b00b-8665f6141000" containerName="revoke-keys"
Sep 16 18:36:47.595629 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.595616    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a6a05b3-a9bb-4c05-b00b-8665f6141000" containerName="revoke-keys"
Sep 16 18:36:47.604131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.604110    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:47.607807 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.607788    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-019b7d43-dockercfg-cwvwg\""
Sep 16 18:36:47.608282 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.608267    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-019b7d43\""
Sep 16 18:36:47.614990 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.614962    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"]
Sep 16 18:36:47.645391 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.645343    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"]
Sep 16 18:36:47.649574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.649552    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:36:47.669650 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.669625    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"]
Sep 16 18:36:47.684112 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.684082    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jg6t\" (UniqueName: \"kubernetes.io/projected/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-kube-api-access-7jg6t\") pod \"payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k\" (UID: \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\") " pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:36:47.684275 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.684133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-plugins-config-volume\") pod \"payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k\" (UID: \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\") " pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:36:47.684275 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.684263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhrzc\" (UniqueName: \"kubernetes.io/projected/47d7b012-69b5-457a-a0c1-62e4cb2be469-kube-api-access-qhrzc\") pod \"payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd\" (UID: \"47d7b012-69b5-457a-a0c1-62e4cb2be469\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:47.684375 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.684331    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47d7b012-69b5-457a-a0c1-62e4cb2be469-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd\" (UID: \"47d7b012-69b5-457a-a0c1-62e4cb2be469\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:47.784925 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.784891    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47d7b012-69b5-457a-a0c1-62e4cb2be469-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd\" (UID: \"47d7b012-69b5-457a-a0c1-62e4cb2be469\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:47.785118 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.784952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jg6t\" (UniqueName: \"kubernetes.io/projected/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-kube-api-access-7jg6t\") pod \"payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k\" (UID: \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\") " pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:36:47.785118 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.784972    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-plugins-config-volume\") pod \"payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k\" (UID: \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\") " pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:36:47.785118 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.785019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhrzc\" (UniqueName: \"kubernetes.io/projected/47d7b012-69b5-457a-a0c1-62e4cb2be469-kube-api-access-qhrzc\") pod \"payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd\" (UID: \"47d7b012-69b5-457a-a0c1-62e4cb2be469\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:47.785772 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.785742    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47d7b012-69b5-457a-a0c1-62e4cb2be469-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd\" (UID: \"47d7b012-69b5-457a-a0c1-62e4cb2be469\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:47.785846 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.785743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-plugins-config-volume\") pod \"payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k\" (UID: \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\") " pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:36:47.794402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.794367    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jg6t\" (UniqueName: \"kubernetes.io/projected/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-kube-api-access-7jg6t\") pod \"payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k\" (UID: \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\") " pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:36:47.797311 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.797285    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhrzc\" (UniqueName: \"kubernetes.io/projected/47d7b012-69b5-457a-a0c1-62e4cb2be469-kube-api-access-qhrzc\") pod \"payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd\" (UID: \"47d7b012-69b5-457a-a0c1-62e4cb2be469\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:47.917955 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.917885    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:47.959598 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:47.959566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:36:48.199679 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.199649    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"]
Sep 16 18:36:48.201425 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:36:48.201405    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47d7b012_69b5_457a_a0c1_62e4cb2be469.slice/crio-6327de34e3a0081c0ca7b55e948122bb2565fc6bd70d9c73b1a5538ea1246c10 WatchSource:0}: Error finding container 6327de34e3a0081c0ca7b55e948122bb2565fc6bd70d9c73b1a5538ea1246c10: Status 404 returned error can't find the container with id 6327de34e3a0081c0ca7b55e948122bb2565fc6bd70d9c73b1a5538ea1246c10
Sep 16 18:36:48.254437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.254411    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"]
Sep 16 18:36:48.254623 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:36:48.254442    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6a04cb_43bd_4d55_a93b_ec284b403b6d.slice/crio-9cc6a464bc81841e8a7a76eda94781690d8916432a332ee194b727a06e018401 WatchSource:0}: Error finding container 9cc6a464bc81841e8a7a76eda94781690d8916432a332ee194b727a06e018401: Status 404 returned error can't find the container with id 9cc6a464bc81841e8a7a76eda94781690d8916432a332ee194b727a06e018401
Sep 16 18:36:48.740657 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.740619    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k" event={"ID":"7c6a04cb-43bd-4d55-a93b-ec284b403b6d","Type":"ContainerStarted","Data":"6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a"}
Sep 16 18:36:48.741143 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.740674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k" event={"ID":"7c6a04cb-43bd-4d55-a93b-ec284b403b6d","Type":"ContainerStarted","Data":"9cc6a464bc81841e8a7a76eda94781690d8916432a332ee194b727a06e018401"}
Sep 16 18:36:48.741143 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.740796    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:36:48.743116 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.743087    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd" event={"ID":"47d7b012-69b5-457a-a0c1-62e4cb2be469","Type":"ContainerStarted","Data":"a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94"}
Sep 16 18:36:48.743312 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.743142    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd" event={"ID":"47d7b012-69b5-457a-a0c1-62e4cb2be469","Type":"ContainerStarted","Data":"6327de34e3a0081c0ca7b55e948122bb2565fc6bd70d9c73b1a5538ea1246c10"}
Sep 16 18:36:48.743312 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.743180    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:48.764870 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.764820    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k" podStartSLOduration=1.764803214 podStartE2EDuration="1.764803214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:36:48.762255932 +0000 UTC m=+1232.318549242" watchObservedRunningTime="2026-09-16 18:36:48.764803214 +0000 UTC m=+1232.321096559"
Sep 16 18:36:48.791695 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:48.791647    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd" podStartSLOduration=1.791631558 podStartE2EDuration="1.791631558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:36:48.790815726 +0000 UTC m=+1232.347109037" watchObservedRunningTime="2026-09-16 18:36:48.791631558 +0000 UTC m=+1232.347924870"
Sep 16 18:36:50.583129 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.583095    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"]
Sep 16 18:36:50.583646 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.583370    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk" podUID="5ca38593-2150-4c57-aa4b-4b1d957aba22" containerName="payload-processing" containerID="cri-o://d4d1ff0f341ce9915b1d4721895e6d86bc9d3e241be92ee05439e91698dfbd8d" gracePeriod=30
Sep 16 18:36:50.594974 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.594945    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"]
Sep 16 18:36:50.595177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.595129    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf" podUID="f40f45c9-42e2-4c92-b6c5-a0eb41590ef4" containerName="payload-pre-processing" containerID="cri-o://6fa62ecf1a0ece9528b253f8ed5a28aad87b7830d9c9d4ebb524336d5fd6bee4" gracePeriod=30
Sep 16 18:36:50.757087 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.756859    2576 generic.go:356] "Generic (PLEG): container finished" podID="f40f45c9-42e2-4c92-b6c5-a0eb41590ef4" containerID="6fa62ecf1a0ece9528b253f8ed5a28aad87b7830d9c9d4ebb524336d5fd6bee4" exitCode=0
Sep 16 18:36:50.757087 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.756987    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf" event={"ID":"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4","Type":"ContainerDied","Data":"6fa62ecf1a0ece9528b253f8ed5a28aad87b7830d9c9d4ebb524336d5fd6bee4"}
Sep 16 18:36:50.759728 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.759701    2576 generic.go:356] "Generic (PLEG): container finished" podID="5ca38593-2150-4c57-aa4b-4b1d957aba22" containerID="d4d1ff0f341ce9915b1d4721895e6d86bc9d3e241be92ee05439e91698dfbd8d" exitCode=0
Sep 16 18:36:50.759866 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.759765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk" event={"ID":"5ca38593-2150-4c57-aa4b-4b1d957aba22","Type":"ContainerDied","Data":"d4d1ff0f341ce9915b1d4721895e6d86bc9d3e241be92ee05439e91698dfbd8d"}
Sep 16 18:36:50.844491 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.844432    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:36:50.864443 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.864416    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"]
Sep 16 18:36:50.878071 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.878044    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-8b875d-66jkh"]
Sep 16 18:36:50.878352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.878335    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:36:50.916839 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.916800    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-plugins-config-volume\") pod \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\" (UID: \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\") "
Sep 16 18:36:50.917027 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.916860    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hn5l\" (UniqueName: \"kubernetes.io/projected/5ca38593-2150-4c57-aa4b-4b1d957aba22-kube-api-access-4hn5l\") pod \"5ca38593-2150-4c57-aa4b-4b1d957aba22\" (UID: \"5ca38593-2150-4c57-aa4b-4b1d957aba22\") "
Sep 16 18:36:50.917027 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.916922    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ca38593-2150-4c57-aa4b-4b1d957aba22-plugins-config-volume\") pod \"5ca38593-2150-4c57-aa4b-4b1d957aba22\" (UID: \"5ca38593-2150-4c57-aa4b-4b1d957aba22\") "
Sep 16 18:36:50.917027 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.917003    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-db58t\" (UniqueName: \"kubernetes.io/projected/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-kube-api-access-db58t\") pod \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\" (UID: \"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4\") "
Sep 16 18:36:50.917315 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.917279    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f40f45c9-42e2-4c92-b6c5-a0eb41590ef4" (UID: "f40f45c9-42e2-4c92-b6c5-a0eb41590ef4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:36:50.917390 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.917339    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ca38593-2150-4c57-aa4b-4b1d957aba22-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5ca38593-2150-4c57-aa4b-4b1d957aba22" (UID: "5ca38593-2150-4c57-aa4b-4b1d957aba22"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:36:50.917501 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.917484    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:36:50.917564 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.917506    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5ca38593-2150-4c57-aa4b-4b1d957aba22-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:36:50.919295 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.919177    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-kube-api-access-db58t" (OuterVolumeSpecName: "kube-api-access-db58t") pod "f40f45c9-42e2-4c92-b6c5-a0eb41590ef4" (UID: "f40f45c9-42e2-4c92-b6c5-a0eb41590ef4"). InnerVolumeSpecName "kube-api-access-db58t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:36:50.919295 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.919189    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ca38593-2150-4c57-aa4b-4b1d957aba22-kube-api-access-4hn5l" (OuterVolumeSpecName: "kube-api-access-4hn5l") pod "5ca38593-2150-4c57-aa4b-4b1d957aba22" (UID: "5ca38593-2150-4c57-aa4b-4b1d957aba22"). InnerVolumeSpecName "kube-api-access-4hn5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:36:50.946271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:50.946234    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a6a05b3-a9bb-4c05-b00b-8665f6141000" path="/var/lib/kubelet/pods/5a6a05b3-a9bb-4c05-b00b-8665f6141000/volumes"
Sep 16 18:36:51.018991 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.018953    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hn5l\" (UniqueName: \"kubernetes.io/projected/5ca38593-2150-4c57-aa4b-4b1d957aba22-kube-api-access-4hn5l\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:36:51.018991 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.018982    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db58t\" (UniqueName: \"kubernetes.io/projected/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4-kube-api-access-db58t\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:36:51.766735 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.766705    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"
Sep 16 18:36:51.767285 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.766734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf" event={"ID":"f40f45c9-42e2-4c92-b6c5-a0eb41590ef4","Type":"ContainerDied","Data":"aaa4e38739c7a2a0183e6fbb4327d91ac6750cf9f15e6b53a4f1ca91a303b6d7"}
Sep 16 18:36:51.767285 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.766792    2576 scope.go:117] "RemoveContainer" containerID="6fa62ecf1a0ece9528b253f8ed5a28aad87b7830d9c9d4ebb524336d5fd6bee4"
Sep 16 18:36:51.768259 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.768238    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk" event={"ID":"5ca38593-2150-4c57-aa4b-4b1d957aba22","Type":"ContainerDied","Data":"c4931d2f1d184fb0e0a17c35a652bbae7a2fc112aa489346d0b83c5e6b9ea882"}
Sep 16 18:36:51.768259 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.768250    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"
Sep 16 18:36:51.776530 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.776514    2576 scope.go:117] "RemoveContainer" containerID="d4d1ff0f341ce9915b1d4721895e6d86bc9d3e241be92ee05439e91698dfbd8d"
Sep 16 18:36:51.784983 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.784958    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"]
Sep 16 18:36:51.800685 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.800659    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf"]
Sep 16 18:36:51.821936 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.821909    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"]
Sep 16 18:36:51.824340 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:51.824321    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk"]
Sep 16 18:36:52.945950 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:52.945918    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ca38593-2150-4c57-aa4b-4b1d957aba22" path="/var/lib/kubelet/pods/5ca38593-2150-4c57-aa4b-4b1d957aba22/volumes"
Sep 16 18:36:52.946362 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:52.946297    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f40f45c9-42e2-4c92-b6c5-a0eb41590ef4" path="/var/lib/kubelet/pods/f40f45c9-42e2-4c92-b6c5-a0eb41590ef4/volumes"
Sep 16 18:36:59.750321 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:59.750287    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:36:59.750748 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:36:59.750446    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:37:27.023246 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.023199    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"]
Sep 16 18:37:27.023778 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.023461    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k" podUID="7c6a04cb-43bd-4d55-a93b-ec284b403b6d" containerName="payload-processing" containerID="cri-o://6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a" gracePeriod=30
Sep 16 18:37:27.040008 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.039979    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"]
Sep 16 18:37:27.040190 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.040165    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd" podUID="47d7b012-69b5-457a-a0c1-62e4cb2be469" containerName="payload-pre-processing" containerID="cri-o://a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94" gracePeriod=30
Sep 16 18:37:27.319408 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.319381    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:37:27.322611 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.322588    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:37:27.384348 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.384318    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jg6t\" (UniqueName: \"kubernetes.io/projected/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-kube-api-access-7jg6t\") pod \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\" (UID: \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\") "
Sep 16 18:37:27.384519 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.384363    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47d7b012-69b5-457a-a0c1-62e4cb2be469-plugins-config-volume\") pod \"47d7b012-69b5-457a-a0c1-62e4cb2be469\" (UID: \"47d7b012-69b5-457a-a0c1-62e4cb2be469\") "
Sep 16 18:37:27.384519 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.384481    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-plugins-config-volume\") pod \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\" (UID: \"7c6a04cb-43bd-4d55-a93b-ec284b403b6d\") "
Sep 16 18:37:27.384650 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.384516    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhrzc\" (UniqueName: \"kubernetes.io/projected/47d7b012-69b5-457a-a0c1-62e4cb2be469-kube-api-access-qhrzc\") pod \"47d7b012-69b5-457a-a0c1-62e4cb2be469\" (UID: \"47d7b012-69b5-457a-a0c1-62e4cb2be469\") "
Sep 16 18:37:27.384989 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.384960    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7c6a04cb-43bd-4d55-a93b-ec284b403b6d" (UID: "7c6a04cb-43bd-4d55-a93b-ec284b403b6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:37:27.385165 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.385017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47d7b012-69b5-457a-a0c1-62e4cb2be469-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "47d7b012-69b5-457a-a0c1-62e4cb2be469" (UID: "47d7b012-69b5-457a-a0c1-62e4cb2be469"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:37:27.386649 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.386625    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47d7b012-69b5-457a-a0c1-62e4cb2be469-kube-api-access-qhrzc" (OuterVolumeSpecName: "kube-api-access-qhrzc") pod "47d7b012-69b5-457a-a0c1-62e4cb2be469" (UID: "47d7b012-69b5-457a-a0c1-62e4cb2be469"). InnerVolumeSpecName "kube-api-access-qhrzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:37:27.387016 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.386992    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-kube-api-access-7jg6t" (OuterVolumeSpecName: "kube-api-access-7jg6t") pod "7c6a04cb-43bd-4d55-a93b-ec284b403b6d" (UID: "7c6a04cb-43bd-4d55-a93b-ec284b403b6d"). InnerVolumeSpecName "kube-api-access-7jg6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:37:27.485498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.485467    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:37:27.485498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.485492    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhrzc\" (UniqueName: \"kubernetes.io/projected/47d7b012-69b5-457a-a0c1-62e4cb2be469-kube-api-access-qhrzc\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:37:27.485498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.485503    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jg6t\" (UniqueName: \"kubernetes.io/projected/7c6a04cb-43bd-4d55-a93b-ec284b403b6d-kube-api-access-7jg6t\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:37:27.485714 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.485512    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/47d7b012-69b5-457a-a0c1-62e4cb2be469-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:37:27.539437 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.539408    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"]
Sep 16 18:37:27.539979 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.539956    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ca38593-2150-4c57-aa4b-4b1d957aba22" containerName="payload-processing"
Sep 16 18:37:27.539979 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.539973    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ca38593-2150-4c57-aa4b-4b1d957aba22" containerName="payload-processing"
Sep 16 18:37:27.540093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.539986    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f40f45c9-42e2-4c92-b6c5-a0eb41590ef4" containerName="payload-pre-processing"
Sep 16 18:37:27.540093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.539992    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f40f45c9-42e2-4c92-b6c5-a0eb41590ef4" containerName="payload-pre-processing"
Sep 16 18:37:27.540093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.540015    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47d7b012-69b5-457a-a0c1-62e4cb2be469" containerName="payload-pre-processing"
Sep 16 18:37:27.540093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.540024    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="47d7b012-69b5-457a-a0c1-62e4cb2be469" containerName="payload-pre-processing"
Sep 16 18:37:27.540093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.540030    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c6a04cb-43bd-4d55-a93b-ec284b403b6d" containerName="payload-processing"
Sep 16 18:37:27.540093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.540035    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c6a04cb-43bd-4d55-a93b-ec284b403b6d" containerName="payload-processing"
Sep 16 18:37:27.540434 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.540101    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ca38593-2150-4c57-aa4b-4b1d957aba22" containerName="payload-processing"
Sep 16 18:37:27.540434 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.540108    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f40f45c9-42e2-4c92-b6c5-a0eb41590ef4" containerName="payload-pre-processing"
Sep 16 18:37:27.540434 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.540115    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c6a04cb-43bd-4d55-a93b-ec284b403b6d" containerName="payload-processing"
Sep 16 18:37:27.540434 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.540127    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="47d7b012-69b5-457a-a0c1-62e4cb2be469" containerName="payload-pre-processing"
Sep 16 18:37:27.543982 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.543934    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.546745 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.546723    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-efa70c-serving-cert\""
Sep 16 18:37:27.546964 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.546946    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 18:37:27.547028 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.546945    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-r78k9\""
Sep 16 18:37:27.561475 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.561450    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"]
Sep 16 18:37:27.586689 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.586666    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-maas-api-tls\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.586811 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.586698    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-metrics-certs\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.586811 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.586792    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqq6s\" (UniqueName: \"kubernetes.io/projected/63a425cc-6865-42c8-ab1b-f54173568201-kube-api-access-nqq6s\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.586911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.586866    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a425cc-6865-42c8-ab1b-f54173568201-openshift-service-ca\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.688035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.688008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-maas-api-tls\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.688179 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.688043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-metrics-certs\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.688179 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.688084    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq6s\" (UniqueName: \"kubernetes.io/projected/63a425cc-6865-42c8-ab1b-f54173568201-kube-api-access-nqq6s\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.688179 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.688143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a425cc-6865-42c8-ab1b-f54173568201-openshift-service-ca\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.688179 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:37:27.688169    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-serving-cert: secret "maas-api-e2e-explicit-efa70c-serving-cert" not found
Sep 16 18:37:27.688412 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:37:27.688275    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-maas-api-tls podName:63a425cc-6865-42c8-ab1b-f54173568201 nodeName:}" failed. No retries permitted until 2026-09-16 18:37:28.188250914 +0000 UTC m=+1271.744544206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-maas-api-tls") pod "maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56" (UID: "63a425cc-6865-42c8-ab1b-f54173568201") : secret "maas-api-e2e-explicit-efa70c-serving-cert" not found
Sep 16 18:37:27.688828 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.688806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a425cc-6865-42c8-ab1b-f54173568201-openshift-service-ca\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.690496 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.690471    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-metrics-certs\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.703767 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.703744    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqq6s\" (UniqueName: \"kubernetes.io/projected/63a425cc-6865-42c8-ab1b-f54173568201-kube-api-access-nqq6s\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:27.977691 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.977664    2576 generic.go:356] "Generic (PLEG): container finished" podID="7c6a04cb-43bd-4d55-a93b-ec284b403b6d" containerID="6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a" exitCode=0
Sep 16 18:37:27.977872 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.977707    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"
Sep 16 18:37:27.977872 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.977744    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k" event={"ID":"7c6a04cb-43bd-4d55-a93b-ec284b403b6d","Type":"ContainerDied","Data":"6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a"}
Sep 16 18:37:27.977872 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.977791    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k" event={"ID":"7c6a04cb-43bd-4d55-a93b-ec284b403b6d","Type":"ContainerDied","Data":"9cc6a464bc81841e8a7a76eda94781690d8916432a332ee194b727a06e018401"}
Sep 16 18:37:27.977872 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.977816    2576 scope.go:117] "RemoveContainer" containerID="6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a"
Sep 16 18:37:27.979199 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.979175    2576 generic.go:356] "Generic (PLEG): container finished" podID="47d7b012-69b5-457a-a0c1-62e4cb2be469" containerID="a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94" exitCode=0
Sep 16 18:37:27.979327 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.979236    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"
Sep 16 18:37:27.979327 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.979260    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd" event={"ID":"47d7b012-69b5-457a-a0c1-62e4cb2be469","Type":"ContainerDied","Data":"a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94"}
Sep 16 18:37:27.979327 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.979316    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd" event={"ID":"47d7b012-69b5-457a-a0c1-62e4cb2be469","Type":"ContainerDied","Data":"6327de34e3a0081c0ca7b55e948122bb2565fc6bd70d9c73b1a5538ea1246c10"}
Sep 16 18:37:27.987562 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.987546    2576 scope.go:117] "RemoveContainer" containerID="6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a"
Sep 16 18:37:27.987807 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:37:27.987792    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a\": container with ID starting with 6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a not found: ID does not exist" containerID="6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a"
Sep 16 18:37:27.987888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.987815    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a"} err="failed to get container status \"6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a\": rpc error: code = NotFound desc = could not find container \"6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a\": container with ID starting with 6ea4a2fbdf24e28d910b01af997d5955a5d15cb5ab751117b49e5c125bc06c0a not found: ID does not exist"
Sep 16 18:37:27.987888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.987830    2576 scope.go:117] "RemoveContainer" containerID="a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94"
Sep 16 18:37:27.996969 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.996949    2576 scope.go:117] "RemoveContainer" containerID="a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94"
Sep 16 18:37:27.997382 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:37:27.997358    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94\": container with ID starting with a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94 not found: ID does not exist" containerID="a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94"
Sep 16 18:37:27.997520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:27.997497    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94"} err="failed to get container status \"a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94\": rpc error: code = NotFound desc = could not find container \"a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94\": container with ID starting with a6f3c097b86cfca79cef556e79612b472259713e7e6b48c80eeb3dfe9fac6b94 not found: ID does not exist"
Sep 16 18:37:28.008495 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.008475    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"]
Sep 16 18:37:28.013385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.013365    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd"]
Sep 16 18:37:28.027345 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.027317    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"]
Sep 16 18:37:28.035270 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.035250    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k"]
Sep 16 18:37:28.192022 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.191991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-maas-api-tls\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:28.194408 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.194386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-maas-api-tls\") pod \"maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:28.354874 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.354783    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"]
Sep 16 18:37:28.359919 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.359897    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:28.362780 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.362762    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-efa70c\""
Sep 16 18:37:28.363357 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.363342    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-efa70c-dockercfg-tnvr5\""
Sep 16 18:37:28.379147 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.379118    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"]
Sep 16 18:37:28.394093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.394070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qpt2\" (UniqueName: \"kubernetes.io/projected/eed59270-2bca-486a-a4c2-206646dcd8cb-kube-api-access-6qpt2\") pod \"payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h\" (UID: \"eed59270-2bca-486a-a4c2-206646dcd8cb\") " pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:28.394237 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.394102    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed59270-2bca-486a-a4c2-206646dcd8cb-plugins-config-volume\") pod \"payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h\" (UID: \"eed59270-2bca-486a-a4c2-206646dcd8cb\") " pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:28.454296 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.454273    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:28.495024 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.494989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qpt2\" (UniqueName: \"kubernetes.io/projected/eed59270-2bca-486a-a4c2-206646dcd8cb-kube-api-access-6qpt2\") pod \"payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h\" (UID: \"eed59270-2bca-486a-a4c2-206646dcd8cb\") " pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:28.495192 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.495028    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed59270-2bca-486a-a4c2-206646dcd8cb-plugins-config-volume\") pod \"payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h\" (UID: \"eed59270-2bca-486a-a4c2-206646dcd8cb\") " pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:28.495749 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.495725    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed59270-2bca-486a-a4c2-206646dcd8cb-plugins-config-volume\") pod \"payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h\" (UID: \"eed59270-2bca-486a-a4c2-206646dcd8cb\") " pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:28.507934 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.507907    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qpt2\" (UniqueName: \"kubernetes.io/projected/eed59270-2bca-486a-a4c2-206646dcd8cb-kube-api-access-6qpt2\") pod \"payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h\" (UID: \"eed59270-2bca-486a-a4c2-206646dcd8cb\") " pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:28.670257 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.670200    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:28.738539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.737888    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"]
Sep 16 18:37:28.739603 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:37:28.739576    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63a425cc_6865_42c8_ab1b_f54173568201.slice/crio-ce78a1031e1a9f134fcee6d7455dc190ab2d9088f0ddf6d9587ae83b98d71a9e WatchSource:0}: Error finding container ce78a1031e1a9f134fcee6d7455dc190ab2d9088f0ddf6d9587ae83b98d71a9e: Status 404 returned error can't find the container with id ce78a1031e1a9f134fcee6d7455dc190ab2d9088f0ddf6d9587ae83b98d71a9e
Sep 16 18:37:28.952636 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.952560    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47d7b012-69b5-457a-a0c1-62e4cb2be469" path="/var/lib/kubelet/pods/47d7b012-69b5-457a-a0c1-62e4cb2be469/volumes"
Sep 16 18:37:28.952982 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.952968    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c6a04cb-43bd-4d55-a93b-ec284b403b6d" path="/var/lib/kubelet/pods/7c6a04cb-43bd-4d55-a93b-ec284b403b6d/volumes"
Sep 16 18:37:28.969643 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.969623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"]
Sep 16 18:37:28.971957 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:37:28.971922    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed59270_2bca_486a_a4c2_206646dcd8cb.slice/crio-a879150c72db1184442975ca6894e507857aa95ed6230f9d9b0cc56159b1d076 WatchSource:0}: Error finding container a879150c72db1184442975ca6894e507857aa95ed6230f9d9b0cc56159b1d076: Status 404 returned error can't find the container with id a879150c72db1184442975ca6894e507857aa95ed6230f9d9b0cc56159b1d076
Sep 16 18:37:28.986231 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.986182    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h" event={"ID":"eed59270-2bca-486a-a4c2-206646dcd8cb","Type":"ContainerStarted","Data":"a879150c72db1184442975ca6894e507857aa95ed6230f9d9b0cc56159b1d076"}
Sep 16 18:37:28.987466 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:28.987435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56" event={"ID":"63a425cc-6865-42c8-ab1b-f54173568201","Type":"ContainerStarted","Data":"ce78a1031e1a9f134fcee6d7455dc190ab2d9088f0ddf6d9587ae83b98d71a9e"}
Sep 16 18:37:29.996501 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:29.996466    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h" event={"ID":"eed59270-2bca-486a-a4c2-206646dcd8cb","Type":"ContainerStarted","Data":"5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc"}
Sep 16 18:37:29.996884 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:29.996600    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:30.022712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:30.022658    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h" podStartSLOduration=2.022641434 podStartE2EDuration="2.022641434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:30.021689465 +0000 UTC m=+1273.577982775" watchObservedRunningTime="2026-09-16 18:37:30.022641434 +0000 UTC m=+1273.578934744"
Sep 16 18:37:31.003918 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:31.003880    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56" event={"ID":"63a425cc-6865-42c8-ab1b-f54173568201","Type":"ContainerStarted","Data":"284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0"}
Sep 16 18:37:31.004375 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:31.004050    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:31.036074 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:31.036024    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56" podStartSLOduration=2.321451457 podStartE2EDuration="4.036005477s" totalImagesPullingTime="1.71455402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:31.034369105 +0000 UTC m=+1274.590662440" watchObservedRunningTime="2026-09-16 18:37:31.036005477 +0000 UTC m=+1274.592298791"
Sep 16 18:37:35.891731 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:35.891695    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"]
Sep 16 18:37:35.895541 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:35.895518    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:37:35.898275 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:35.898246    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-b95e3be6-dockercfg-mtfjp\""
Sep 16 18:37:35.898477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:35.898462    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-b95e3be6\""
Sep 16 18:37:35.915486 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:35.915462    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"]
Sep 16 18:37:35.965884 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:35.965856    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ffbfb83-1202-4069-8e77-5e57a355d745-plugins-config-volume\") pod \"payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj\" (UID: \"0ffbfb83-1202-4069-8e77-5e57a355d745\") " pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:37:35.966058 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:35.965923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmss2\" (UniqueName: \"kubernetes.io/projected/0ffbfb83-1202-4069-8e77-5e57a355d745-kube-api-access-mmss2\") pod \"payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj\" (UID: \"0ffbfb83-1202-4069-8e77-5e57a355d745\") " pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:37:36.067409 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:36.067373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ffbfb83-1202-4069-8e77-5e57a355d745-plugins-config-volume\") pod \"payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj\" (UID: \"0ffbfb83-1202-4069-8e77-5e57a355d745\") " pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:37:36.067584 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:36.067457    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmss2\" (UniqueName: \"kubernetes.io/projected/0ffbfb83-1202-4069-8e77-5e57a355d745-kube-api-access-mmss2\") pod \"payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj\" (UID: \"0ffbfb83-1202-4069-8e77-5e57a355d745\") " pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:37:36.068335 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:36.068313    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ffbfb83-1202-4069-8e77-5e57a355d745-plugins-config-volume\") pod \"payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj\" (UID: \"0ffbfb83-1202-4069-8e77-5e57a355d745\") " pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:37:36.079937 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:36.079910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmss2\" (UniqueName: \"kubernetes.io/projected/0ffbfb83-1202-4069-8e77-5e57a355d745-kube-api-access-mmss2\") pod \"payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj\" (UID: \"0ffbfb83-1202-4069-8e77-5e57a355d745\") " pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:37:36.205629 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:36.205558    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:37:36.493203 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:36.493177    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"]
Sep 16 18:37:36.495457 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:37:36.495422    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ffbfb83_1202_4069_8e77_5e57a355d745.slice/crio-4c4e1aa22785ec767bb58a10798311f06b3e1a3637e2b189dde7d5d17a2fa9e5 WatchSource:0}: Error finding container 4c4e1aa22785ec767bb58a10798311f06b3e1a3637e2b189dde7d5d17a2fa9e5: Status 404 returned error can't find the container with id 4c4e1aa22785ec767bb58a10798311f06b3e1a3637e2b189dde7d5d17a2fa9e5
Sep 16 18:37:37.015779 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:37.015746    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:37:37.049925 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:37.049875    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj" event={"ID":"0ffbfb83-1202-4069-8e77-5e57a355d745","Type":"ContainerStarted","Data":"9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c"}
Sep 16 18:37:37.050094 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:37.049952    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj" event={"ID":"0ffbfb83-1202-4069-8e77-5e57a355d745","Type":"ContainerStarted","Data":"4c4e1aa22785ec767bb58a10798311f06b3e1a3637e2b189dde7d5d17a2fa9e5"}
Sep 16 18:37:37.050094 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:37.050057    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:37:37.088950 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:37.088903    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj" podStartSLOduration=2.088887825 podStartE2EDuration="2.088887825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:37:37.087918234 +0000 UTC m=+1280.644211548" watchObservedRunningTime="2026-09-16 18:37:37.088887825 +0000 UTC m=+1280.645181136"
Sep 16 18:37:41.006014 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:41.005979    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:37:48.058105 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:37:48.058069    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:38:03.444371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:03.444337    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"]
Sep 16 18:38:03.444895 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:03.444549    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj" podUID="0ffbfb83-1202-4069-8e77-5e57a355d745" containerName="payload-processing" containerID="cri-o://9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c" gracePeriod=30
Sep 16 18:38:03.702551 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:03.702487    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:38:03.728330 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:03.728296    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmss2\" (UniqueName: \"kubernetes.io/projected/0ffbfb83-1202-4069-8e77-5e57a355d745-kube-api-access-mmss2\") pod \"0ffbfb83-1202-4069-8e77-5e57a355d745\" (UID: \"0ffbfb83-1202-4069-8e77-5e57a355d745\") "
Sep 16 18:38:03.728505 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:03.728439    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ffbfb83-1202-4069-8e77-5e57a355d745-plugins-config-volume\") pod \"0ffbfb83-1202-4069-8e77-5e57a355d745\" (UID: \"0ffbfb83-1202-4069-8e77-5e57a355d745\") "
Sep 16 18:38:03.728868 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:03.728794    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ffbfb83-1202-4069-8e77-5e57a355d745-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0ffbfb83-1202-4069-8e77-5e57a355d745" (UID: "0ffbfb83-1202-4069-8e77-5e57a355d745"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:38:03.730948 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:03.730918    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ffbfb83-1202-4069-8e77-5e57a355d745-kube-api-access-mmss2" (OuterVolumeSpecName: "kube-api-access-mmss2") pod "0ffbfb83-1202-4069-8e77-5e57a355d745" (UID: "0ffbfb83-1202-4069-8e77-5e57a355d745"). InnerVolumeSpecName "kube-api-access-mmss2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:38:03.829847 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:03.829818    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmss2\" (UniqueName: \"kubernetes.io/projected/0ffbfb83-1202-4069-8e77-5e57a355d745-kube-api-access-mmss2\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:38:03.829847 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:03.829844    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ffbfb83-1202-4069-8e77-5e57a355d745-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:38:04.207665 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.207630    2576 generic.go:356] "Generic (PLEG): container finished" podID="0ffbfb83-1202-4069-8e77-5e57a355d745" containerID="9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c" exitCode=0
Sep 16 18:38:04.207856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.207677    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"
Sep 16 18:38:04.207856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.207716    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj" event={"ID":"0ffbfb83-1202-4069-8e77-5e57a355d745","Type":"ContainerDied","Data":"9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c"}
Sep 16 18:38:04.207856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.207764    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj" event={"ID":"0ffbfb83-1202-4069-8e77-5e57a355d745","Type":"ContainerDied","Data":"4c4e1aa22785ec767bb58a10798311f06b3e1a3637e2b189dde7d5d17a2fa9e5"}
Sep 16 18:38:04.207856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.207787    2576 scope.go:117] "RemoveContainer" containerID="9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c"
Sep 16 18:38:04.217444 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.217429    2576 scope.go:117] "RemoveContainer" containerID="9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c"
Sep 16 18:38:04.217696 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:38:04.217679    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c\": container with ID starting with 9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c not found: ID does not exist" containerID="9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c"
Sep 16 18:38:04.217742 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.217704    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c"} err="failed to get container status \"9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c\": rpc error: code = NotFound desc = could not find container \"9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c\": container with ID starting with 9a14712412e55bbf69f6afd95b2be9007dfd769f2f4f42f3dd638ff84b0bd71c not found: ID does not exist"
Sep 16 18:38:04.247291 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.247265    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"]
Sep 16 18:38:04.252052 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.252030    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj"]
Sep 16 18:38:04.945420 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:04.945386    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ffbfb83-1202-4069-8e77-5e57a355d745" path="/var/lib/kubelet/pods/0ffbfb83-1202-4069-8e77-5e57a355d745/volumes"
Sep 16 18:38:46.889525 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.889448    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"]
Sep 16 18:38:46.890008 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.889993    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ffbfb83-1202-4069-8e77-5e57a355d745" containerName="payload-processing"
Sep 16 18:38:46.890053 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.890010    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ffbfb83-1202-4069-8e77-5e57a355d745" containerName="payload-processing"
Sep 16 18:38:46.890098 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.890089    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ffbfb83-1202-4069-8e77-5e57a355d745" containerName="payload-processing"
Sep 16 18:38:46.893134 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.893114    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:46.899345 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.899327    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-efa70c\"/\"default-dockercfg-cnbph\""
Sep 16 18:38:46.899490 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.899354    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-efa70c\"/\"openshift-service-ca.crt\""
Sep 16 18:38:46.900408 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.900200    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-efa70c\"/\"kube-root-ca.crt\""
Sep 16 18:38:46.900522 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.900499    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-efa70c\"/\"explicit-model-efa70c-kserve-self-signed-certs\""
Sep 16 18:38:46.946574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:46.946551    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"]
Sep 16 18:38:47.012007 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.011974    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-tmp-dir\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.012176 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.012052    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-dshm\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.012176 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.012092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjfn6\" (UniqueName: \"kubernetes.io/projected/086ca4be-485a-480c-971e-b460ea603535-kube-api-access-mjfn6\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.012176 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.012116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-home\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.012176 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.012160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-model-cache\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.012404 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.012258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086ca4be-485a-480c-971e-b460ea603535-tls-certs\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.113270 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.113237    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjfn6\" (UniqueName: \"kubernetes.io/projected/086ca4be-485a-480c-971e-b460ea603535-kube-api-access-mjfn6\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.113270 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.113273    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-home\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.113498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.113316    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-model-cache\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.113498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.113364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086ca4be-485a-480c-971e-b460ea603535-tls-certs\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.113498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.113440    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-tmp-dir\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.113498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.113495    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-dshm\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.113770 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.113707    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-home\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.113770 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.113740    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-model-cache\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.113848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.113775    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-tmp-dir\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.115693 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.115665    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-dshm\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.115857 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.115839    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086ca4be-485a-480c-971e-b460ea603535-tls-certs\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.140296 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.140236    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjfn6\" (UniqueName: \"kubernetes.io/projected/086ca4be-485a-480c-971e-b460ea603535-kube-api-access-mjfn6\") pod \"explicit-model-efa70c-kserve-6f4f7d7745-44k9g\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") " pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.202572 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.202542    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:47.497123 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:47.497095    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"]
Sep 16 18:38:47.499140 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:38:47.499112    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod086ca4be_485a_480c_971e_b460ea603535.slice/crio-2bb96975d3e55130fd05044d755fcfae55984d6359ef1cd6a57abb9f38ac3bd4 WatchSource:0}: Error finding container 2bb96975d3e55130fd05044d755fcfae55984d6359ef1cd6a57abb9f38ac3bd4: Status 404 returned error can't find the container with id 2bb96975d3e55130fd05044d755fcfae55984d6359ef1cd6a57abb9f38ac3bd4
Sep 16 18:38:48.444531 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:48.444497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g" event={"ID":"086ca4be-485a-480c-971e-b460ea603535","Type":"ContainerStarted","Data":"e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92"}
Sep 16 18:38:48.444531 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:48.444538    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g" event={"ID":"086ca4be-485a-480c-971e-b460ea603535","Type":"ContainerStarted","Data":"2bb96975d3e55130fd05044d755fcfae55984d6359ef1cd6a57abb9f38ac3bd4"}
Sep 16 18:38:48.467302 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:48.467260    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g" podStartSLOduration=2.467245151 podStartE2EDuration="2.467245151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:38:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:38:48.466330177 +0000 UTC m=+1352.022623518" watchObservedRunningTime="2026-09-16 18:38:48.467245151 +0000 UTC m=+1352.023538476"
Sep 16 18:38:57.203277 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:57.203246    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:57.203633 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:57.203290    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:57.215907 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:57.215883    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:38:57.503361 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:38:57.503287    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:39:13.181473 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:13.180244    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"]
Sep 16 18:39:24.054421 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.054389    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"]
Sep 16 18:39:24.054966 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.054602    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56" podUID="63a425cc-6865-42c8-ab1b-f54173568201" containerName="maas-api" containerID="cri-o://284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0" gracePeriod=30
Sep 16 18:39:24.132830 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.132798    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"]
Sep 16 18:39:24.133062 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.133037    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h" podUID="eed59270-2bca-486a-a4c2-206646dcd8cb" containerName="payload-processing" containerID="cri-o://5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc" gracePeriod=30
Sep 16 18:39:24.393082 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.393058    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:39:24.418198 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.418172    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:39:24.464632 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.464592    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6qpt2\" (UniqueName: \"kubernetes.io/projected/eed59270-2bca-486a-a4c2-206646dcd8cb-kube-api-access-6qpt2\") pod \"eed59270-2bca-486a-a4c2-206646dcd8cb\" (UID: \"eed59270-2bca-486a-a4c2-206646dcd8cb\") "
Sep 16 18:39:24.464822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.464643    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqq6s\" (UniqueName: \"kubernetes.io/projected/63a425cc-6865-42c8-ab1b-f54173568201-kube-api-access-nqq6s\") pod \"63a425cc-6865-42c8-ab1b-f54173568201\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") "
Sep 16 18:39:24.464822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.464672    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed59270-2bca-486a-a4c2-206646dcd8cb-plugins-config-volume\") pod \"eed59270-2bca-486a-a4c2-206646dcd8cb\" (UID: \"eed59270-2bca-486a-a4c2-206646dcd8cb\") "
Sep 16 18:39:24.464822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.464745    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a425cc-6865-42c8-ab1b-f54173568201-openshift-service-ca\") pod \"63a425cc-6865-42c8-ab1b-f54173568201\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") "
Sep 16 18:39:24.464822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.464780    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-maas-api-tls\") pod \"63a425cc-6865-42c8-ab1b-f54173568201\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") "
Sep 16 18:39:24.464822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.464810    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-metrics-certs\") pod \"63a425cc-6865-42c8-ab1b-f54173568201\" (UID: \"63a425cc-6865-42c8-ab1b-f54173568201\") "
Sep 16 18:39:24.465131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.465100    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eed59270-2bca-486a-a4c2-206646dcd8cb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eed59270-2bca-486a-a4c2-206646dcd8cb" (UID: "eed59270-2bca-486a-a4c2-206646dcd8cb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:39:24.465193 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.465130    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63a425cc-6865-42c8-ab1b-f54173568201-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63a425cc-6865-42c8-ab1b-f54173568201" (UID: "63a425cc-6865-42c8-ab1b-f54173568201"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:39:24.465278 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.465222    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a425cc-6865-42c8-ab1b-f54173568201-openshift-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:24.465278 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.465239    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eed59270-2bca-486a-a4c2-206646dcd8cb-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:24.466852 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.466827    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eed59270-2bca-486a-a4c2-206646dcd8cb-kube-api-access-6qpt2" (OuterVolumeSpecName: "kube-api-access-6qpt2") pod "eed59270-2bca-486a-a4c2-206646dcd8cb" (UID: "eed59270-2bca-486a-a4c2-206646dcd8cb"). InnerVolumeSpecName "kube-api-access-6qpt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:39:24.467105 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.467070    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "63a425cc-6865-42c8-ab1b-f54173568201" (UID: "63a425cc-6865-42c8-ab1b-f54173568201"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:39:24.467410 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.467382    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63a425cc-6865-42c8-ab1b-f54173568201-kube-api-access-nqq6s" (OuterVolumeSpecName: "kube-api-access-nqq6s") pod "63a425cc-6865-42c8-ab1b-f54173568201" (UID: "63a425cc-6865-42c8-ab1b-f54173568201"). InnerVolumeSpecName "kube-api-access-nqq6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:39:24.467410 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.467379    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63a425cc-6865-42c8-ab1b-f54173568201" (UID: "63a425cc-6865-42c8-ab1b-f54173568201"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:39:24.566306 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.566273    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qpt2\" (UniqueName: \"kubernetes.io/projected/eed59270-2bca-486a-a4c2-206646dcd8cb-kube-api-access-6qpt2\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:24.566306 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.566300    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqq6s\" (UniqueName: \"kubernetes.io/projected/63a425cc-6865-42c8-ab1b-f54173568201-kube-api-access-nqq6s\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:24.566306 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.566310    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-maas-api-tls\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:24.566531 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.566319    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63a425cc-6865-42c8-ab1b-f54173568201-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:24.638526 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.638449    2576 generic.go:356] "Generic (PLEG): container finished" podID="eed59270-2bca-486a-a4c2-206646dcd8cb" containerID="5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc" exitCode=0
Sep 16 18:39:24.638526 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.638495    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"
Sep 16 18:39:24.638723 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.638535    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h" event={"ID":"eed59270-2bca-486a-a4c2-206646dcd8cb","Type":"ContainerDied","Data":"5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc"}
Sep 16 18:39:24.638723 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.638595    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h" event={"ID":"eed59270-2bca-486a-a4c2-206646dcd8cb","Type":"ContainerDied","Data":"a879150c72db1184442975ca6894e507857aa95ed6230f9d9b0cc56159b1d076"}
Sep 16 18:39:24.638723 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.638628    2576 scope.go:117] "RemoveContainer" containerID="5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc"
Sep 16 18:39:24.640074 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.640048    2576 generic.go:356] "Generic (PLEG): container finished" podID="63a425cc-6865-42c8-ab1b-f54173568201" containerID="284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0" exitCode=0
Sep 16 18:39:24.640169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.640079    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"
Sep 16 18:39:24.640169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.640130    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56" event={"ID":"63a425cc-6865-42c8-ab1b-f54173568201","Type":"ContainerDied","Data":"284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0"}
Sep 16 18:39:24.640318 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.640187    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56" event={"ID":"63a425cc-6865-42c8-ab1b-f54173568201","Type":"ContainerDied","Data":"ce78a1031e1a9f134fcee6d7455dc190ab2d9088f0ddf6d9587ae83b98d71a9e"}
Sep 16 18:39:24.649292 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.649271    2576 scope.go:117] "RemoveContainer" containerID="5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc"
Sep 16 18:39:24.649709 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:39:24.649589    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc\": container with ID starting with 5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc not found: ID does not exist" containerID="5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc"
Sep 16 18:39:24.649709 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.649624    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc"} err="failed to get container status \"5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc\": rpc error: code = NotFound desc = could not find container \"5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc\": container with ID starting with 5b534e919f71caeb4123f8b2484d42017ac188062075707b4c1c3351d229f1dc not found: ID does not exist"
Sep 16 18:39:24.649709 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.649647    2576 scope.go:117] "RemoveContainer" containerID="284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0"
Sep 16 18:39:24.660181 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.660162    2576 scope.go:117] "RemoveContainer" containerID="284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0"
Sep 16 18:39:24.660507 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:39:24.660488    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0\": container with ID starting with 284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0 not found: ID does not exist" containerID="284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0"
Sep 16 18:39:24.660561 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.660514    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0"} err="failed to get container status \"284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0\": rpc error: code = NotFound desc = could not find container \"284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0\": container with ID starting with 284c384d6d89c3ae5c7ce08a2d686cab1c6769280cde1731db588fdccb4d58c0 not found: ID does not exist"
Sep 16 18:39:24.672678 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.671813    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"]
Sep 16 18:39:24.675852 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.675829    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h"]
Sep 16 18:39:24.681866 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.681848    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"]
Sep 16 18:39:24.685979 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.685959    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-efa70c-5dbc7ccbb4-h7z56"]
Sep 16 18:39:24.945033 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.945002    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63a425cc-6865-42c8-ab1b-f54173568201" path="/var/lib/kubelet/pods/63a425cc-6865-42c8-ab1b-f54173568201/volumes"
Sep 16 18:39:24.945444 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:24.945429    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eed59270-2bca-486a-a4c2-206646dcd8cb" path="/var/lib/kubelet/pods/eed59270-2bca-486a-a4c2-206646dcd8cb/volumes"
Sep 16 18:39:28.204891 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.204837    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g" podUID="086ca4be-485a-480c-971e-b460ea603535" containerName="main" containerID="cri-o://e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92" gracePeriod=45
Sep 16 18:39:28.461294 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.461242    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:39:28.507628 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.507600    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-dshm\") pod \"086ca4be-485a-480c-971e-b460ea603535\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") "
Sep 16 18:39:28.507729 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.507692    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-home\") pod \"086ca4be-485a-480c-971e-b460ea603535\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") "
Sep 16 18:39:28.507729 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.507718    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-model-cache\") pod \"086ca4be-485a-480c-971e-b460ea603535\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") "
Sep 16 18:39:28.507848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.507782    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086ca4be-485a-480c-971e-b460ea603535-tls-certs\") pod \"086ca4be-485a-480c-971e-b460ea603535\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") "
Sep 16 18:39:28.507848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.507808    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-tmp-dir\") pod \"086ca4be-485a-480c-971e-b460ea603535\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") "
Sep 16 18:39:28.507848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.507845    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjfn6\" (UniqueName: \"kubernetes.io/projected/086ca4be-485a-480c-971e-b460ea603535-kube-api-access-mjfn6\") pod \"086ca4be-485a-480c-971e-b460ea603535\" (UID: \"086ca4be-485a-480c-971e-b460ea603535\") "
Sep 16 18:39:28.508027 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.507968    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-home" (OuterVolumeSpecName: "home") pod "086ca4be-485a-480c-971e-b460ea603535" (UID: "086ca4be-485a-480c-971e-b460ea603535"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:39:28.508163 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.508043    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-model-cache" (OuterVolumeSpecName: "model-cache") pod "086ca4be-485a-480c-971e-b460ea603535" (UID: "086ca4be-485a-480c-971e-b460ea603535"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:39:28.508163 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.508141    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "086ca4be-485a-480c-971e-b460ea603535" (UID: "086ca4be-485a-480c-971e-b460ea603535"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:39:28.508383 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.508366    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-model-cache\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:28.508383 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.508384    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-tmp-dir\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:28.508474 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.508392    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-home\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:28.510428 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.510350    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/086ca4be-485a-480c-971e-b460ea603535-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "086ca4be-485a-480c-971e-b460ea603535" (UID: "086ca4be-485a-480c-971e-b460ea603535"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:39:28.510530 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.510442    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-dshm" (OuterVolumeSpecName: "dshm") pod "086ca4be-485a-480c-971e-b460ea603535" (UID: "086ca4be-485a-480c-971e-b460ea603535"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:39:28.510530 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.510449    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/086ca4be-485a-480c-971e-b460ea603535-kube-api-access-mjfn6" (OuterVolumeSpecName: "kube-api-access-mjfn6") pod "086ca4be-485a-480c-971e-b460ea603535" (UID: "086ca4be-485a-480c-971e-b460ea603535"). InnerVolumeSpecName "kube-api-access-mjfn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:39:28.609675 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.609646    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/086ca4be-485a-480c-971e-b460ea603535-dshm\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:28.609675 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.609666    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/086ca4be-485a-480c-971e-b460ea603535-tls-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:28.609675 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.609675    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjfn6\" (UniqueName: \"kubernetes.io/projected/086ca4be-485a-480c-971e-b460ea603535-kube-api-access-mjfn6\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:39:28.665752 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.665721    2576 generic.go:356] "Generic (PLEG): container finished" podID="086ca4be-485a-480c-971e-b460ea603535" containerID="e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92" exitCode=0
Sep 16 18:39:28.665879 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.665769    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"
Sep 16 18:39:28.665879 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.665803    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g" event={"ID":"086ca4be-485a-480c-971e-b460ea603535","Type":"ContainerDied","Data":"e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92"}
Sep 16 18:39:28.665879 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.665851    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g" event={"ID":"086ca4be-485a-480c-971e-b460ea603535","Type":"ContainerDied","Data":"2bb96975d3e55130fd05044d755fcfae55984d6359ef1cd6a57abb9f38ac3bd4"}
Sep 16 18:39:28.665879 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.665870    2576 scope.go:117] "RemoveContainer" containerID="e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92"
Sep 16 18:39:28.675013 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.674998    2576 scope.go:117] "RemoveContainer" containerID="e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92"
Sep 16 18:39:28.675306 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:39:28.675287    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92\": container with ID starting with e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92 not found: ID does not exist" containerID="e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92"
Sep 16 18:39:28.675382 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.675318    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92"} err="failed to get container status \"e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92\": rpc error: code = NotFound desc = could not find container \"e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92\": container with ID starting with e020d0df57d9fe84cf119307f948358a5992c2fb887b270822bd58c278d65e92 not found: ID does not exist"
Sep 16 18:39:28.698350 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.698326    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"]
Sep 16 18:39:28.703395 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.703376    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-efa70c/explicit-model-efa70c-kserve-6f4f7d7745-44k9g"]
Sep 16 18:39:28.945515 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:28.945489    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="086ca4be-485a-480c-971e-b460ea603535" path="/var/lib/kubelet/pods/086ca4be-485a-480c-971e-b460ea603535/volumes"
Sep 16 18:39:54.132856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.132825    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"]
Sep 16 18:39:54.133324 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.133309    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63a425cc-6865-42c8-ab1b-f54173568201" containerName="maas-api"
Sep 16 18:39:54.133324 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.133325    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="63a425cc-6865-42c8-ab1b-f54173568201" containerName="maas-api"
Sep 16 18:39:54.133441 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.133339    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="086ca4be-485a-480c-971e-b460ea603535" containerName="main"
Sep 16 18:39:54.133441 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.133344    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="086ca4be-485a-480c-971e-b460ea603535" containerName="main"
Sep 16 18:39:54.133441 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.133354    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eed59270-2bca-486a-a4c2-206646dcd8cb" containerName="payload-processing"
Sep 16 18:39:54.133441 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.133360    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eed59270-2bca-486a-a4c2-206646dcd8cb" containerName="payload-processing"
Sep 16 18:39:54.133588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.133485    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="63a425cc-6865-42c8-ab1b-f54173568201" containerName="maas-api"
Sep 16 18:39:54.133588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.133496    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="086ca4be-485a-480c-971e-b460ea603535" containerName="main"
Sep 16 18:39:54.133588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.133505    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eed59270-2bca-486a-a4c2-206646dcd8cb" containerName="payload-processing"
Sep 16 18:39:54.137009 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.136980    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.139248 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.139193    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-58beb43a-serving-cert\""
Sep 16 18:39:54.139351 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.139284    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 18:39:54.140025 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.140005    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-r78k9\""
Sep 16 18:39:54.152191 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.152172    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"]
Sep 16 18:39:54.236185 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.236153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-openshift-service-ca\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.236323 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.236239    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gd27\" (UniqueName: \"kubernetes.io/projected/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-kube-api-access-5gd27\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.236323 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.236268    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-metrics-certs\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.236323 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.236311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-maas-api-tls\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.336998 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.336970    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gd27\" (UniqueName: \"kubernetes.io/projected/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-kube-api-access-5gd27\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.337127 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.337003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-metrics-certs\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.337173 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.337136    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-maas-api-tls\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.337270 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.337251    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-openshift-service-ca\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.337316 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:39:54.337264    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-serving-cert: secret "maas-api-e2e-mt-58beb43a-serving-cert" not found
Sep 16 18:39:54.337353 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:39:54.337343    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-maas-api-tls podName:4f84527b-ed7a-4f02-af92-9cd8947ab2b9 nodeName:}" failed. No retries permitted until 2026-09-16 18:39:54.837324024 +0000 UTC m=+1418.393617317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-maas-api-tls") pod "maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd" (UID: "4f84527b-ed7a-4f02-af92-9cd8947ab2b9") : secret "maas-api-e2e-mt-58beb43a-serving-cert" not found
Sep 16 18:39:54.337848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.337821    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-openshift-service-ca\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.339426 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.339411    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-metrics-certs\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.350722 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.350705    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gd27\" (UniqueName: \"kubernetes.io/projected/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-kube-api-access-5gd27\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.842808 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.842779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-maas-api-tls\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:54.845095 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:54.845075    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-maas-api-tls\") pod \"maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:55.047591 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.047560    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:55.146031 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.146002    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"]
Sep 16 18:39:55.150178 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.150163    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:39:55.152649 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.152628    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-58beb43a\""
Sep 16 18:39:55.152765 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.152700    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-58beb43a-dockercfg-xcx6g\""
Sep 16 18:39:55.162332 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.162306    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"]
Sep 16 18:39:55.202920 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.202894    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"]
Sep 16 18:39:55.206254 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.206233    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:39:55.231808 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.231778    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"]
Sep 16 18:39:55.246589 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.246566    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjjvf\" (UniqueName: \"kubernetes.io/projected/d8255337-8b04-4288-9971-6c47394db08e-kube-api-access-xjjvf\") pod \"payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh\" (UID: \"d8255337-8b04-4288-9971-6c47394db08e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:39:55.246725 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.246702    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8255337-8b04-4288-9971-6c47394db08e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh\" (UID: \"d8255337-8b04-4288-9971-6c47394db08e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:39:55.328444 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.328421    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"]
Sep 16 18:39:55.331605 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:39:55.331582    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f84527b_ed7a_4f02_af92_9cd8947ab2b9.slice/crio-26766897fc5031bcc8133ac8c1bab74c0c3468c83903f1a7d45168b5e449cb58 WatchSource:0}: Error finding container 26766897fc5031bcc8133ac8c1bab74c0c3468c83903f1a7d45168b5e449cb58: Status 404 returned error can't find the container with id 26766897fc5031bcc8133ac8c1bab74c0c3468c83903f1a7d45168b5e449cb58
Sep 16 18:39:55.332988 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.332970    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:39:55.348134 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.348064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjjvf\" (UniqueName: \"kubernetes.io/projected/d8255337-8b04-4288-9971-6c47394db08e-kube-api-access-xjjvf\") pod \"payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh\" (UID: \"d8255337-8b04-4288-9971-6c47394db08e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:39:55.348251 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.348162    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6l9v\" (UniqueName: \"kubernetes.io/projected/05231d57-c2e2-44e5-9c42-f2b7ed799fac-kube-api-access-p6l9v\") pod \"payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd\" (UID: \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\") " pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:39:55.348251 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.348203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8255337-8b04-4288-9971-6c47394db08e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh\" (UID: \"d8255337-8b04-4288-9971-6c47394db08e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:39:55.348431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.348339    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05231d57-c2e2-44e5-9c42-f2b7ed799fac-plugins-config-volume\") pod \"payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd\" (UID: \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\") " pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:39:55.349095 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.349075    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8255337-8b04-4288-9971-6c47394db08e-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh\" (UID: \"d8255337-8b04-4288-9971-6c47394db08e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:39:55.357782 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.357759    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjjvf\" (UniqueName: \"kubernetes.io/projected/d8255337-8b04-4288-9971-6c47394db08e-kube-api-access-xjjvf\") pod \"payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh\" (UID: \"d8255337-8b04-4288-9971-6c47394db08e\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:39:55.449236 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.449188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05231d57-c2e2-44e5-9c42-f2b7ed799fac-plugins-config-volume\") pod \"payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd\" (UID: \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\") " pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:39:55.449351 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.449317    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6l9v\" (UniqueName: \"kubernetes.io/projected/05231d57-c2e2-44e5-9c42-f2b7ed799fac-kube-api-access-p6l9v\") pod \"payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd\" (UID: \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\") " pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:39:55.449822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.449802    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05231d57-c2e2-44e5-9c42-f2b7ed799fac-plugins-config-volume\") pod \"payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd\" (UID: \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\") " pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:39:55.457609 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.457584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6l9v\" (UniqueName: \"kubernetes.io/projected/05231d57-c2e2-44e5-9c42-f2b7ed799fac-kube-api-access-p6l9v\") pod \"payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd\" (UID: \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\") " pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:39:55.462271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.462250    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:39:55.516847 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.516816    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:39:55.738641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.738616    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"]
Sep 16 18:39:55.740637 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:39:55.740611    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8255337_8b04_4288_9971_6c47394db08e.slice/crio-eec2f7421e614877faf4fa210c17562af678ea07fe45b3631f05e4fdfef3411e WatchSource:0}: Error finding container eec2f7421e614877faf4fa210c17562af678ea07fe45b3631f05e4fdfef3411e: Status 404 returned error can't find the container with id eec2f7421e614877faf4fa210c17562af678ea07fe45b3631f05e4fdfef3411e
Sep 16 18:39:55.803848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.803810    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"]
Sep 16 18:39:55.805681 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:39:55.805659    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05231d57_c2e2_44e5_9c42_f2b7ed799fac.slice/crio-e2fe0b41fb2493fcbf0a40c204971b264bd012322f25e1077b6ab5bd6774eb31 WatchSource:0}: Error finding container e2fe0b41fb2493fcbf0a40c204971b264bd012322f25e1077b6ab5bd6774eb31: Status 404 returned error can't find the container with id e2fe0b41fb2493fcbf0a40c204971b264bd012322f25e1077b6ab5bd6774eb31
Sep 16 18:39:55.809197 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.809168    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh" event={"ID":"d8255337-8b04-4288-9971-6c47394db08e","Type":"ContainerStarted","Data":"eec2f7421e614877faf4fa210c17562af678ea07fe45b3631f05e4fdfef3411e"}
Sep 16 18:39:55.810932 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.810905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd" event={"ID":"4f84527b-ed7a-4f02-af92-9cd8947ab2b9","Type":"ContainerStarted","Data":"703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a"}
Sep 16 18:39:55.811057 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.810953    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd" event={"ID":"4f84527b-ed7a-4f02-af92-9cd8947ab2b9","Type":"ContainerStarted","Data":"26766897fc5031bcc8133ac8c1bab74c0c3468c83903f1a7d45168b5e449cb58"}
Sep 16 18:39:55.812360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.812339    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:39:55.836604 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:55.836556    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd" podStartSLOduration=1.465464119 podStartE2EDuration="1.836539731s" totalImagesPullingTime="371.075612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:55.836331429 +0000 UTC m=+1419.392624741" watchObservedRunningTime="2026-09-16 18:39:55.836539731 +0000 UTC m=+1419.392833041"
Sep 16 18:39:56.819239 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:56.819189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh" event={"ID":"d8255337-8b04-4288-9971-6c47394db08e","Type":"ContainerStarted","Data":"15138ec2964c9127626845fb162332b73eb9807195d14e29d772177d1f5abbfa"}
Sep 16 18:39:56.819673 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:56.819369    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:39:56.820894 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:56.820868    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd" event={"ID":"05231d57-c2e2-44e5-9c42-f2b7ed799fac","Type":"ContainerStarted","Data":"8e61da3d27fb975b329477d08f4a885d3cd575f8f3a555b29fb56f6541ef7c44"}
Sep 16 18:39:56.821030 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:56.820906    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd" event={"ID":"05231d57-c2e2-44e5-9c42-f2b7ed799fac","Type":"ContainerStarted","Data":"e2fe0b41fb2493fcbf0a40c204971b264bd012322f25e1077b6ab5bd6774eb31"}
Sep 16 18:39:56.821101 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:56.821086    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:39:56.844803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:56.844757    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh" podStartSLOduration=1.844744312 podStartE2EDuration="1.844744312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:56.841472622 +0000 UTC m=+1420.397765938" watchObservedRunningTime="2026-09-16 18:39:56.844744312 +0000 UTC m=+1420.401037670"
Sep 16 18:39:56.865584 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:39:56.865541    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd" podStartSLOduration=1.865528727 podStartE2EDuration="1.865528727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:39:56.863070079 +0000 UTC m=+1420.419363394" watchObservedRunningTime="2026-09-16 18:39:56.865528727 +0000 UTC m=+1420.421822071"
Sep 16 18:40:01.826537 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:01.826506    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:40:07.833187 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:07.833159    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:40:07.833576 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:07.833237    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:40:11.239226 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.239130    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"]
Sep 16 18:40:11.243734 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.243717    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.250292 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.250236    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-909a40\"/\"kube-root-ca.crt\""
Sep 16 18:40:11.251004 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.250968    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-909a40\"/\"default-dockercfg-rqmcw\""
Sep 16 18:40:11.255009 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.251676    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-909a40\"/\"openshift-service-ca.crt\""
Sep 16 18:40:11.255009 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.252437    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-909a40\"/\"distinct-w0-909a40-kserve-self-signed-certs\""
Sep 16 18:40:11.263266 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.263236    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"]
Sep 16 18:40:11.280762 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.280737    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-model-cache\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.280864 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.280765    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-tmp-dir\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.280864 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.280834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-dshm\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.280943 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.280877    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcs27\" (UniqueName: \"kubernetes.io/projected/e48e7cb1-245c-4da0-9650-13188bcc50a0-kube-api-access-tcs27\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.280943 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.280924    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e48e7cb1-245c-4da0-9650-13188bcc50a0-tls-certs\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.281009 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.280967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-home\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.382071 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.382049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-model-cache\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.382177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.382078    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-tmp-dir\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.382177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.382113    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-dshm\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.382177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.382137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcs27\" (UniqueName: \"kubernetes.io/projected/e48e7cb1-245c-4da0-9650-13188bcc50a0-kube-api-access-tcs27\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.382177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.382174    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e48e7cb1-245c-4da0-9650-13188bcc50a0-tls-certs\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.382382 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.382191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-home\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.382524 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.382502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-model-cache\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.382591 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.382544    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-tmp-dir\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.382591 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.382576    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-home\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.384414 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.384394    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-dshm\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.384707 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.384691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e48e7cb1-245c-4da0-9650-13188bcc50a0-tls-certs\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.391897 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.391877    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcs27\" (UniqueName: \"kubernetes.io/projected/e48e7cb1-245c-4da0-9650-13188bcc50a0-kube-api-access-tcs27\") pod \"distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") " pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.556148 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.556091    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:11.832303 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.832280    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"]
Sep 16 18:40:11.833922 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:40:11.833898    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode48e7cb1_245c_4da0_9650_13188bcc50a0.slice/crio-09114d439c087b8175ce7e8a69f39b0c8ac54be0103a69de5ec32be58c71764c WatchSource:0}: Error finding container 09114d439c087b8175ce7e8a69f39b0c8ac54be0103a69de5ec32be58c71764c: Status 404 returned error can't find the container with id 09114d439c087b8175ce7e8a69f39b0c8ac54be0103a69de5ec32be58c71764c
Sep 16 18:40:11.924811 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.924781    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg" event={"ID":"e48e7cb1-245c-4da0-9650-13188bcc50a0","Type":"ContainerStarted","Data":"7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9"}
Sep 16 18:40:11.924939 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.924833    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg" event={"ID":"e48e7cb1-245c-4da0-9650-13188bcc50a0","Type":"ContainerStarted","Data":"09114d439c087b8175ce7e8a69f39b0c8ac54be0103a69de5ec32be58c71764c"}
Sep 16 18:40:11.943536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:11.943492    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg" podStartSLOduration=0.943477391 podStartE2EDuration="943.477391ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:11.9427889 +0000 UTC m=+1435.499082224" watchObservedRunningTime="2026-09-16 18:40:11.943477391 +0000 UTC m=+1435.499770702"
Sep 16 18:40:21.557236 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:21.557186    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:21.557650 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:21.557257    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:21.569813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:21.569789    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:21.992342 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:21.992319    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:40:40.585352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:40.585322    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"]
Sep 16 18:40:40.585752 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:40.585508    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd" podUID="4f84527b-ed7a-4f02-af92-9cd8947ab2b9" containerName="maas-api" containerID="cri-o://703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a" gracePeriod=30
Sep 16 18:40:40.782961 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:40.782644    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"]
Sep 16 18:40:40.782961 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:40.782889    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd" podUID="05231d57-c2e2-44e5-9c42-f2b7ed799fac" containerName="payload-processing" containerID="cri-o://8e61da3d27fb975b329477d08f4a885d3cd575f8f3a555b29fb56f6541ef7c44" gracePeriod=30
Sep 16 18:40:40.831129 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:40.831099    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"]
Sep 16 18:40:40.831373 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:40.831331    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh" podUID="d8255337-8b04-4288-9971-6c47394db08e" containerName="payload-pre-processing" containerID="cri-o://15138ec2964c9127626845fb162332b73eb9807195d14e29d772177d1f5abbfa" gracePeriod=30
Sep 16 18:40:40.943096 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:40.943065    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:40:41.057771 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.057742    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gd27\" (UniqueName: \"kubernetes.io/projected/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-kube-api-access-5gd27\") pod \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") "
Sep 16 18:40:41.057917 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.057778    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-openshift-service-ca\") pod \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") "
Sep 16 18:40:41.057917 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.057800    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-maas-api-tls\") pod \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") "
Sep 16 18:40:41.058045 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.057925    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-metrics-certs\") pod \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\" (UID: \"4f84527b-ed7a-4f02-af92-9cd8947ab2b9\") "
Sep 16 18:40:41.058563 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.058535    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4f84527b-ed7a-4f02-af92-9cd8947ab2b9" (UID: "4f84527b-ed7a-4f02-af92-9cd8947ab2b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:40:41.060239 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.060197    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4f84527b-ed7a-4f02-af92-9cd8947ab2b9" (UID: "4f84527b-ed7a-4f02-af92-9cd8947ab2b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:40:41.060356 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.060247    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4f84527b-ed7a-4f02-af92-9cd8947ab2b9" (UID: "4f84527b-ed7a-4f02-af92-9cd8947ab2b9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:40:41.060600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.060577    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-kube-api-access-5gd27" (OuterVolumeSpecName: "kube-api-access-5gd27") pod "4f84527b-ed7a-4f02-af92-9cd8947ab2b9" (UID: "4f84527b-ed7a-4f02-af92-9cd8947ab2b9"). InnerVolumeSpecName "kube-api-access-5gd27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:40:41.087334 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.087307    2576 generic.go:356] "Generic (PLEG): container finished" podID="d8255337-8b04-4288-9971-6c47394db08e" containerID="15138ec2964c9127626845fb162332b73eb9807195d14e29d772177d1f5abbfa" exitCode=0
Sep 16 18:40:41.087471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.087379    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh" event={"ID":"d8255337-8b04-4288-9971-6c47394db08e","Type":"ContainerDied","Data":"15138ec2964c9127626845fb162332b73eb9807195d14e29d772177d1f5abbfa"}
Sep 16 18:40:41.088699 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.088678    2576 generic.go:356] "Generic (PLEG): container finished" podID="4f84527b-ed7a-4f02-af92-9cd8947ab2b9" containerID="703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a" exitCode=0
Sep 16 18:40:41.088805 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.088735    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"
Sep 16 18:40:41.088805 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.088761    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd" event={"ID":"4f84527b-ed7a-4f02-af92-9cd8947ab2b9","Type":"ContainerDied","Data":"703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a"}
Sep 16 18:40:41.088930 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.088819    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd" event={"ID":"4f84527b-ed7a-4f02-af92-9cd8947ab2b9","Type":"ContainerDied","Data":"26766897fc5031bcc8133ac8c1bab74c0c3468c83903f1a7d45168b5e449cb58"}
Sep 16 18:40:41.088930 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.088851    2576 scope.go:117] "RemoveContainer" containerID="703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a"
Sep 16 18:40:41.090735 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.090710    2576 generic.go:356] "Generic (PLEG): container finished" podID="05231d57-c2e2-44e5-9c42-f2b7ed799fac" containerID="8e61da3d27fb975b329477d08f4a885d3cd575f8f3a555b29fb56f6541ef7c44" exitCode=0
Sep 16 18:40:41.090850 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.090753    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd" event={"ID":"05231d57-c2e2-44e5-9c42-f2b7ed799fac","Type":"ContainerDied","Data":"8e61da3d27fb975b329477d08f4a885d3cd575f8f3a555b29fb56f6541ef7c44"}
Sep 16 18:40:41.100536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.100512    2576 scope.go:117] "RemoveContainer" containerID="703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a"
Sep 16 18:40:41.100857 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:40:41.100828    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a\": container with ID starting with 703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a not found: ID does not exist" containerID="703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a"
Sep 16 18:40:41.100953 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.100856    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a"} err="failed to get container status \"703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a\": rpc error: code = NotFound desc = could not find container \"703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a\": container with ID starting with 703685436c1d28f5eb9f239b067ff8f0c74976220f1055ef82c1ab530215f77a not found: ID does not exist"
Sep 16 18:40:41.110273 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.110250    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:40:41.112242 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.112224    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:40:41.148181 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.148154    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"]
Sep 16 18:40:41.157558 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.157536    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-58beb43a-7b95d5845c-k2cvd"]
Sep 16 18:40:41.159096 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.159072    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:40:41.159226 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.159101    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gd27\" (UniqueName: \"kubernetes.io/projected/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-kube-api-access-5gd27\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:40:41.159226 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.159114    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-openshift-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:40:41.159226 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.159126    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f84527b-ed7a-4f02-af92-9cd8947ab2b9-maas-api-tls\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:40:41.260274 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.260244    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05231d57-c2e2-44e5-9c42-f2b7ed799fac-plugins-config-volume\") pod \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\" (UID: \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\") "
Sep 16 18:40:41.260474 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.260322    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6l9v\" (UniqueName: \"kubernetes.io/projected/05231d57-c2e2-44e5-9c42-f2b7ed799fac-kube-api-access-p6l9v\") pod \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\" (UID: \"05231d57-c2e2-44e5-9c42-f2b7ed799fac\") "
Sep 16 18:40:41.260474 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.260379    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjjvf\" (UniqueName: \"kubernetes.io/projected/d8255337-8b04-4288-9971-6c47394db08e-kube-api-access-xjjvf\") pod \"d8255337-8b04-4288-9971-6c47394db08e\" (UID: \"d8255337-8b04-4288-9971-6c47394db08e\") "
Sep 16 18:40:41.260615 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.260487    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8255337-8b04-4288-9971-6c47394db08e-plugins-config-volume\") pod \"d8255337-8b04-4288-9971-6c47394db08e\" (UID: \"d8255337-8b04-4288-9971-6c47394db08e\") "
Sep 16 18:40:41.260675 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.260654    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05231d57-c2e2-44e5-9c42-f2b7ed799fac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "05231d57-c2e2-44e5-9c42-f2b7ed799fac" (UID: "05231d57-c2e2-44e5-9c42-f2b7ed799fac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:40:41.261135 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.261100    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8255337-8b04-4288-9971-6c47394db08e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d8255337-8b04-4288-9971-6c47394db08e" (UID: "d8255337-8b04-4288-9971-6c47394db08e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:40:41.261300 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.261271    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/05231d57-c2e2-44e5-9c42-f2b7ed799fac-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:40:41.262876 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.262848    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05231d57-c2e2-44e5-9c42-f2b7ed799fac-kube-api-access-p6l9v" (OuterVolumeSpecName: "kube-api-access-p6l9v") pod "05231d57-c2e2-44e5-9c42-f2b7ed799fac" (UID: "05231d57-c2e2-44e5-9c42-f2b7ed799fac"). InnerVolumeSpecName "kube-api-access-p6l9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:40:41.263240 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.263199    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8255337-8b04-4288-9971-6c47394db08e-kube-api-access-xjjvf" (OuterVolumeSpecName: "kube-api-access-xjjvf") pod "d8255337-8b04-4288-9971-6c47394db08e" (UID: "d8255337-8b04-4288-9971-6c47394db08e"). InnerVolumeSpecName "kube-api-access-xjjvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:40:41.362431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.362375    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjjvf\" (UniqueName: \"kubernetes.io/projected/d8255337-8b04-4288-9971-6c47394db08e-kube-api-access-xjjvf\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:40:41.362431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.362395    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8255337-8b04-4288-9971-6c47394db08e-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:40:41.362431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:41.362406    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6l9v\" (UniqueName: \"kubernetes.io/projected/05231d57-c2e2-44e5-9c42-f2b7ed799fac-kube-api-access-p6l9v\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:40:42.097885 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.097851    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"
Sep 16 18:40:42.097885 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.097860    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh" event={"ID":"d8255337-8b04-4288-9971-6c47394db08e","Type":"ContainerDied","Data":"eec2f7421e614877faf4fa210c17562af678ea07fe45b3631f05e4fdfef3411e"}
Sep 16 18:40:42.098363 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.097925    2576 scope.go:117] "RemoveContainer" containerID="15138ec2964c9127626845fb162332b73eb9807195d14e29d772177d1f5abbfa"
Sep 16 18:40:42.100485 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.100453    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd" event={"ID":"05231d57-c2e2-44e5-9c42-f2b7ed799fac","Type":"ContainerDied","Data":"e2fe0b41fb2493fcbf0a40c204971b264bd012322f25e1077b6ab5bd6774eb31"}
Sep 16 18:40:42.100485 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.100482    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"
Sep 16 18:40:42.107844 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.107828    2576 scope.go:117] "RemoveContainer" containerID="8e61da3d27fb975b329477d08f4a885d3cd575f8f3a555b29fb56f6541ef7c44"
Sep 16 18:40:42.140600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.140576    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"]
Sep 16 18:40:42.152521 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.152499    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd"]
Sep 16 18:40:42.181893 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.181873    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"]
Sep 16 18:40:42.183766 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.183748    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh"]
Sep 16 18:40:42.945105 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.945065    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05231d57-c2e2-44e5-9c42-f2b7ed799fac" path="/var/lib/kubelet/pods/05231d57-c2e2-44e5-9c42-f2b7ed799fac/volumes"
Sep 16 18:40:42.945498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.945482    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f84527b-ed7a-4f02-af92-9cd8947ab2b9" path="/var/lib/kubelet/pods/4f84527b-ed7a-4f02-af92-9cd8947ab2b9/volumes"
Sep 16 18:40:42.945807 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:40:42.945794    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8255337-8b04-4288-9971-6c47394db08e" path="/var/lib/kubelet/pods/d8255337-8b04-4288-9971-6c47394db08e/volumes"
Sep 16 18:41:04.947323 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.947291    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"]
Sep 16 18:41:04.947946 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.947927    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f84527b-ed7a-4f02-af92-9cd8947ab2b9" containerName="maas-api"
Sep 16 18:41:04.948039 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.947949    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f84527b-ed7a-4f02-af92-9cd8947ab2b9" containerName="maas-api"
Sep 16 18:41:04.948039 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.947961    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05231d57-c2e2-44e5-9c42-f2b7ed799fac" containerName="payload-processing"
Sep 16 18:41:04.948039 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.947969    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="05231d57-c2e2-44e5-9c42-f2b7ed799fac" containerName="payload-processing"
Sep 16 18:41:04.948039 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.947983    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8255337-8b04-4288-9971-6c47394db08e" containerName="payload-pre-processing"
Sep 16 18:41:04.948039 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.947991    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8255337-8b04-4288-9971-6c47394db08e" containerName="payload-pre-processing"
Sep 16 18:41:04.948324 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.948094    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f84527b-ed7a-4f02-af92-9cd8947ab2b9" containerName="maas-api"
Sep 16 18:41:04.948324 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.948107    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8255337-8b04-4288-9971-6c47394db08e" containerName="payload-pre-processing"
Sep 16 18:41:04.948324 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.948119    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="05231d57-c2e2-44e5-9c42-f2b7ed799fac" containerName="payload-processing"
Sep 16 18:41:04.951582 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.951562    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"
Sep 16 18:41:04.954679 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.954658    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-cgwrl\""
Sep 16 18:41:04.966726 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:04.966703    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"]
Sep 16 18:41:05.088827 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:05.088790    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28b8260-3638-49e1-b052-311214af66ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b\" (UID: \"e28b8260-3638-49e1-b052-311214af66ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"
Sep 16 18:41:05.189431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:05.189402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28b8260-3638-49e1-b052-311214af66ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b\" (UID: \"e28b8260-3638-49e1-b052-311214af66ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"
Sep 16 18:41:05.190070 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:05.190047    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28b8260-3638-49e1-b052-311214af66ce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b\" (UID: \"e28b8260-3638-49e1-b052-311214af66ce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"
Sep 16 18:41:05.264763 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:05.264695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"
Sep 16 18:41:05.551339 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:05.551313    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"]
Sep 16 18:41:05.552732 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:41:05.552693    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28b8260_3638_49e1_b052_311214af66ce.slice/crio-54faece49b07d93cbab957a2ad67dda52cf51dd7d145e788ba52c4b7baf84226 WatchSource:0}: Error finding container 54faece49b07d93cbab957a2ad67dda52cf51dd7d145e788ba52c4b7baf84226: Status 404 returned error can't find the container with id 54faece49b07d93cbab957a2ad67dda52cf51dd7d145e788ba52c4b7baf84226
Sep 16 18:41:06.232388 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:06.232353    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b" event={"ID":"e28b8260-3638-49e1-b052-311214af66ce","Type":"ContainerStarted","Data":"93b6ab267947034fbbbf19ecc3e6f4db4af6086ab589625d0dc4394203ea0577"}
Sep 16 18:41:06.232773 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:06.232413    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b" event={"ID":"e28b8260-3638-49e1-b052-311214af66ce","Type":"ContainerStarted","Data":"54faece49b07d93cbab957a2ad67dda52cf51dd7d145e788ba52c4b7baf84226"}
Sep 16 18:41:07.239894 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:07.239860    2576 generic.go:356] "Generic (PLEG): container finished" podID="e28b8260-3638-49e1-b052-311214af66ce" containerID="93b6ab267947034fbbbf19ecc3e6f4db4af6086ab589625d0dc4394203ea0577" exitCode=0
Sep 16 18:41:07.240278 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:07.239922    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b" event={"ID":"e28b8260-3638-49e1-b052-311214af66ce","Type":"ContainerDied","Data":"93b6ab267947034fbbbf19ecc3e6f4db4af6086ab589625d0dc4394203ea0577"}
Sep 16 18:41:08.377926 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:08.377902    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"
Sep 16 18:41:08.522709 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:08.522633    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28b8260-3638-49e1-b052-311214af66ce-maas-api-service-ca\") pod \"e28b8260-3638-49e1-b052-311214af66ce\" (UID: \"e28b8260-3638-49e1-b052-311214af66ce\") "
Sep 16 18:41:08.523086 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:08.523062    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e28b8260-3638-49e1-b052-311214af66ce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e28b8260-3638-49e1-b052-311214af66ce" (UID: "e28b8260-3638-49e1-b052-311214af66ce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:41:08.623799 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:08.623767    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e28b8260-3638-49e1-b052-311214af66ce-maas-api-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:41:09.251293 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:09.251203    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"
Sep 16 18:41:09.251293 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:09.251253    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b" event={"ID":"e28b8260-3638-49e1-b052-311214af66ce","Type":"ContainerDied","Data":"54faece49b07d93cbab957a2ad67dda52cf51dd7d145e788ba52c4b7baf84226"}
Sep 16 18:41:09.251469 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:09.251301    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54faece49b07d93cbab957a2ad67dda52cf51dd7d145e788ba52c4b7baf84226"
Sep 16 18:41:16.699550 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:16.699512    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"]
Sep 16 18:41:16.699950 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:16.699716    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww" podUID="bdb7db43-f916-4c9a-bfe4-894a4deb2047" containerName="payload-processing" containerID="cri-o://ca68de78d1af91e8654c0aa749fffbd2abf3339f7d02a02a6911125a275d1784" gracePeriod=30
Sep 16 18:41:16.757863 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:16.757834    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"]
Sep 16 18:41:16.758068 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:16.758030    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c" podUID="55b96ade-9967-419b-9944-337be017cfc3" containerName="payload-pre-processing" containerID="cri-o://537e08d5a28378a4a0668e7e4f59783d689a338c544e329f2b9968050fb503c3" gracePeriod=30
Sep 16 18:41:17.053610 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.053582    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:41:17.057579 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.057558    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:41:17.096963 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.096935    2576 scope.go:117] "RemoveContainer" containerID="ca68de78d1af91e8654c0aa749fffbd2abf3339f7d02a02a6911125a275d1784"
Sep 16 18:41:17.107667 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.107650    2576 scope.go:117] "RemoveContainer" containerID="537e08d5a28378a4a0668e7e4f59783d689a338c544e329f2b9968050fb503c3"
Sep 16 18:41:17.213246 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.213203    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b96ade-9967-419b-9944-337be017cfc3-plugins-config-volume\") pod \"55b96ade-9967-419b-9944-337be017cfc3\" (UID: \"55b96ade-9967-419b-9944-337be017cfc3\") "
Sep 16 18:41:17.213385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.213309    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb7db43-f916-4c9a-bfe4-894a4deb2047-plugins-config-volume\") pod \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\" (UID: \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\") "
Sep 16 18:41:17.213448 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.213393    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ghvz\" (UniqueName: \"kubernetes.io/projected/55b96ade-9967-419b-9944-337be017cfc3-kube-api-access-8ghvz\") pod \"55b96ade-9967-419b-9944-337be017cfc3\" (UID: \"55b96ade-9967-419b-9944-337be017cfc3\") "
Sep 16 18:41:17.213448 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.213411    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5zh6g\" (UniqueName: \"kubernetes.io/projected/bdb7db43-f916-4c9a-bfe4-894a4deb2047-kube-api-access-5zh6g\") pod \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\" (UID: \"bdb7db43-f916-4c9a-bfe4-894a4deb2047\") "
Sep 16 18:41:17.213680 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.213650    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55b96ade-9967-419b-9944-337be017cfc3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "55b96ade-9967-419b-9944-337be017cfc3" (UID: "55b96ade-9967-419b-9944-337be017cfc3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:41:17.213680 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.213666    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdb7db43-f916-4c9a-bfe4-894a4deb2047-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bdb7db43-f916-4c9a-bfe4-894a4deb2047" (UID: "bdb7db43-f916-4c9a-bfe4-894a4deb2047"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:41:17.215467 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.215409    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55b96ade-9967-419b-9944-337be017cfc3-kube-api-access-8ghvz" (OuterVolumeSpecName: "kube-api-access-8ghvz") pod "55b96ade-9967-419b-9944-337be017cfc3" (UID: "55b96ade-9967-419b-9944-337be017cfc3"). InnerVolumeSpecName "kube-api-access-8ghvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:41:17.215560 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.215506    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdb7db43-f916-4c9a-bfe4-894a4deb2047-kube-api-access-5zh6g" (OuterVolumeSpecName: "kube-api-access-5zh6g") pod "bdb7db43-f916-4c9a-bfe4-894a4deb2047" (UID: "bdb7db43-f916-4c9a-bfe4-894a4deb2047"). InnerVolumeSpecName "kube-api-access-5zh6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:41:17.296493 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.296459    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww" event={"ID":"bdb7db43-f916-4c9a-bfe4-894a4deb2047","Type":"ContainerDied","Data":"ca68de78d1af91e8654c0aa749fffbd2abf3339f7d02a02a6911125a275d1784"}
Sep 16 18:41:17.296637 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.296510    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"
Sep 16 18:41:17.296637 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.296616    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww" event={"ID":"bdb7db43-f916-4c9a-bfe4-894a4deb2047","Type":"ContainerDied","Data":"49f7af5329acf63bddac2033af7ed042e2922825579c7e431b11b255cb47940b"}
Sep 16 18:41:17.296778 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.296652    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c" event={"ID":"55b96ade-9967-419b-9944-337be017cfc3","Type":"ContainerDied","Data":"537e08d5a28378a4a0668e7e4f59783d689a338c544e329f2b9968050fb503c3"}
Sep 16 18:41:17.296778 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.296510    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"
Sep 16 18:41:17.296778 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.296697    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c" event={"ID":"55b96ade-9967-419b-9944-337be017cfc3","Type":"ContainerDied","Data":"5c6bf831132fcf573ad5a578dd8d38c0753979b05df4c2a0f9f262639721dc48"}
Sep 16 18:41:17.314478 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.314459    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ghvz\" (UniqueName: \"kubernetes.io/projected/55b96ade-9967-419b-9944-337be017cfc3-kube-api-access-8ghvz\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:41:17.314478 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.314480    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zh6g\" (UniqueName: \"kubernetes.io/projected/bdb7db43-f916-4c9a-bfe4-894a4deb2047-kube-api-access-5zh6g\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:41:17.314612 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.314489    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/55b96ade-9967-419b-9944-337be017cfc3-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:41:17.314612 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.314498    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bdb7db43-f916-4c9a-bfe4-894a4deb2047-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:41:17.444828 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.444794    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"]
Sep 16 18:41:17.459512 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.459486    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4ab744-xxj9b"]
Sep 16 18:41:17.480631 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.480565    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"]
Sep 16 18:41:17.497438 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.497406    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c"]
Sep 16 18:41:17.521864 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.521839    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"]
Sep 16 18:41:17.550091 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:17.550069    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww"]
Sep 16 18:41:18.945670 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:18.945628    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55b96ade-9967-419b-9944-337be017cfc3" path="/var/lib/kubelet/pods/55b96ade-9967-419b-9944-337be017cfc3/volumes"
Sep 16 18:41:18.946169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:18.946148    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdb7db43-f916-4c9a-bfe4-894a4deb2047" path="/var/lib/kubelet/pods/bdb7db43-f916-4c9a-bfe4-894a4deb2047/volumes"
Sep 16 18:41:18.946611 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:18.946590    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e28b8260-3638-49e1-b052-311214af66ce" path="/var/lib/kubelet/pods/e28b8260-3638-49e1-b052-311214af66ce/volumes"
Sep 16 18:41:23.674191 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674154    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"]
Sep 16 18:41:23.674813 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674795    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdb7db43-f916-4c9a-bfe4-894a4deb2047" containerName="payload-processing"
Sep 16 18:41:23.674861 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674815    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdb7db43-f916-4c9a-bfe4-894a4deb2047" containerName="payload-processing"
Sep 16 18:41:23.674861 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674824    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55b96ade-9967-419b-9944-337be017cfc3" containerName="payload-pre-processing"
Sep 16 18:41:23.674861 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674831    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="55b96ade-9967-419b-9944-337be017cfc3" containerName="payload-pre-processing"
Sep 16 18:41:23.674861 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674841    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28b8260-3638-49e1-b052-311214af66ce" containerName="revoke-keys"
Sep 16 18:41:23.674861 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674846    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28b8260-3638-49e1-b052-311214af66ce" containerName="revoke-keys"
Sep 16 18:41:23.675018 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674927    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="55b96ade-9967-419b-9944-337be017cfc3" containerName="payload-pre-processing"
Sep 16 18:41:23.675018 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674938    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdb7db43-f916-4c9a-bfe4-894a4deb2047" containerName="payload-processing"
Sep 16 18:41:23.675018 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.674944    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28b8260-3638-49e1-b052-311214af66ce" containerName="revoke-keys"
Sep 16 18:41:23.679408 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.679389    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.695825 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.695801    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-04899e-serving-cert\""
Sep 16 18:41:23.716407 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.716380    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-r78k9\""
Sep 16 18:41:23.717181 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.717162    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 18:41:23.736467 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.736440    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"]
Sep 16 18:41:23.775370 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.775341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.775543 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.775466    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/553108f0-a26a-4a43-a0a3-406c8493b4ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.775543 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.775505    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkqlr\" (UniqueName: \"kubernetes.io/projected/553108f0-a26a-4a43-a0a3-406c8493b4ca-kube-api-access-tkqlr\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.775543 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.775539    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.876047 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.876006    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/553108f0-a26a-4a43-a0a3-406c8493b4ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.876047 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.876048    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkqlr\" (UniqueName: \"kubernetes.io/projected/553108f0-a26a-4a43-a0a3-406c8493b4ca-kube-api-access-tkqlr\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.876336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.876087    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.876336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.876282    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.876780 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.876757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/553108f0-a26a-4a43-a0a3-406c8493b4ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.878639 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.878620    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.878639 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.878637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.904405 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.904376    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkqlr\" (UniqueName: \"kubernetes.io/projected/553108f0-a26a-4a43-a0a3-406c8493b4ca-kube-api-access-tkqlr\") pod \"maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:23.989753 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:23.989663    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:24.290576 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:24.290503    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"]
Sep 16 18:41:24.293768 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:41:24.293740    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod553108f0_a26a_4a43_a0a3_406c8493b4ca.slice/crio-6ab5fe11c87f71fb5359b8c8a8b75624919b6d006a5101a3e3699eb725b4010f WatchSource:0}: Error finding container 6ab5fe11c87f71fb5359b8c8a8b75624919b6d006a5101a3e3699eb725b4010f: Status 404 returned error can't find the container with id 6ab5fe11c87f71fb5359b8c8a8b75624919b6d006a5101a3e3699eb725b4010f
Sep 16 18:41:24.333203 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:24.333170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc" event={"ID":"553108f0-a26a-4a43-a0a3-406c8493b4ca","Type":"ContainerStarted","Data":"6ab5fe11c87f71fb5359b8c8a8b75624919b6d006a5101a3e3699eb725b4010f"}
Sep 16 18:41:25.272531 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.272493    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"]
Sep 16 18:41:25.276358 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.276335    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:25.278969 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.278948    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-04899e\""
Sep 16 18:41:25.288647 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.288624    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9\" (UID: \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:25.288763 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.288690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrmxg\" (UniqueName: \"kubernetes.io/projected/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-kube-api-access-hrmxg\") pod \"payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9\" (UID: \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:25.313177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.313149    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"]
Sep 16 18:41:25.340601 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.340569    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc" event={"ID":"553108f0-a26a-4a43-a0a3-406c8493b4ca","Type":"ContainerStarted","Data":"08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003"}
Sep 16 18:41:25.340748 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.340685    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:25.389198 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.389166    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9\" (UID: \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:25.389367 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.389263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrmxg\" (UniqueName: \"kubernetes.io/projected/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-kube-api-access-hrmxg\") pod \"payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9\" (UID: \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:25.389862 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.389844    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9\" (UID: \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:25.406797 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.406755    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc" podStartSLOduration=2.049127563 podStartE2EDuration="2.406740957s" totalImagesPullingTime="357.613394ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:25.396165445 +0000 UTC m=+1508.952458768" watchObservedRunningTime="2026-09-16 18:41:25.406740957 +0000 UTC m=+1508.963034267"
Sep 16 18:41:25.422681 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.422661    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrmxg\" (UniqueName: \"kubernetes.io/projected/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-kube-api-access-hrmxg\") pod \"payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9\" (UID: \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:25.586771 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.586695    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:25.891806 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:25.891768    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"]
Sep 16 18:41:25.895087 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:41:25.895055    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4ab3d54_066d_44c0_b3a4_f2ef19cf4f5a.slice/crio-ccb8052e8d5aebcc2301f6ea44ed6715a7a6d155ceb429916e97ef66bc23c634 WatchSource:0}: Error finding container ccb8052e8d5aebcc2301f6ea44ed6715a7a6d155ceb429916e97ef66bc23c634: Status 404 returned error can't find the container with id ccb8052e8d5aebcc2301f6ea44ed6715a7a6d155ceb429916e97ef66bc23c634
Sep 16 18:41:26.348898 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:26.348857    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9" event={"ID":"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a","Type":"ContainerStarted","Data":"75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d"}
Sep 16 18:41:26.349311 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:26.348939    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9" event={"ID":"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a","Type":"ContainerStarted","Data":"ccb8052e8d5aebcc2301f6ea44ed6715a7a6d155ceb429916e97ef66bc23c634"}
Sep 16 18:41:26.384774 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:26.384718    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9" podStartSLOduration=1.384700182 podStartE2EDuration="1.384700182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:26.382235081 +0000 UTC m=+1509.938528388" watchObservedRunningTime="2026-09-16 18:41:26.384700182 +0000 UTC m=+1509.940993537"
Sep 16 18:41:27.355413 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:27.355381    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:31.354360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:31.354333    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:41:38.363642 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:38.363616    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:41:45.364123 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.364028    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"]
Sep 16 18:41:45.364620 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.364530    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl" podUID="88590f47-c61b-4e8b-b871-c479ccc641cf" containerName="payload-processing" containerID="cri-o://05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027" gracePeriod=30
Sep 16 18:41:45.376360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.376150    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"]
Sep 16 18:41:45.376728 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.376462    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm" podUID="2e2f74f1-d75b-44fa-9ebb-12c0b566796b" containerName="payload-pre-processing" containerID="cri-o://e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958" gracePeriod=30
Sep 16 18:41:45.655430 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.655408    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:41:45.658142 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.658122    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:41:45.768087 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.768048    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"]
Sep 16 18:41:45.768536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.768521    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e2f74f1-d75b-44fa-9ebb-12c0b566796b" containerName="payload-pre-processing"
Sep 16 18:41:45.768590 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.768538    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e2f74f1-d75b-44fa-9ebb-12c0b566796b" containerName="payload-pre-processing"
Sep 16 18:41:45.768590 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.768559    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88590f47-c61b-4e8b-b871-c479ccc641cf" containerName="payload-processing"
Sep 16 18:41:45.768590 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.768564    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="88590f47-c61b-4e8b-b871-c479ccc641cf" containerName="payload-processing"
Sep 16 18:41:45.768689 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.768632    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e2f74f1-d75b-44fa-9ebb-12c0b566796b" containerName="payload-pre-processing"
Sep 16 18:41:45.768689 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.768644    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="88590f47-c61b-4e8b-b871-c479ccc641cf" containerName="payload-processing"
Sep 16 18:41:45.771721 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.771698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.778146 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.778128    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-520999-serving-cert\""
Sep 16 18:41:45.793068 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.793040    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkjp4\" (UniqueName: \"kubernetes.io/projected/88590f47-c61b-4e8b-b871-c479ccc641cf-kube-api-access-pkjp4\") pod \"88590f47-c61b-4e8b-b871-c479ccc641cf\" (UID: \"88590f47-c61b-4e8b-b871-c479ccc641cf\") "
Sep 16 18:41:45.793237 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.793146    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88590f47-c61b-4e8b-b871-c479ccc641cf-plugins-config-volume\") pod \"88590f47-c61b-4e8b-b871-c479ccc641cf\" (UID: \"88590f47-c61b-4e8b-b871-c479ccc641cf\") "
Sep 16 18:41:45.793331 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.793239    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj75m\" (UniqueName: \"kubernetes.io/projected/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-kube-api-access-tj75m\") pod \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\" (UID: \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\") "
Sep 16 18:41:45.793331 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.793296    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-plugins-config-volume\") pod \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\" (UID: \"2e2f74f1-d75b-44fa-9ebb-12c0b566796b\") "
Sep 16 18:41:45.793757 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.793732    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88590f47-c61b-4e8b-b871-c479ccc641cf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "88590f47-c61b-4e8b-b871-c479ccc641cf" (UID: "88590f47-c61b-4e8b-b871-c479ccc641cf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:41:45.793898 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.793862    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2e2f74f1-d75b-44fa-9ebb-12c0b566796b" (UID: "2e2f74f1-d75b-44fa-9ebb-12c0b566796b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:41:45.795661 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.795627    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88590f47-c61b-4e8b-b871-c479ccc641cf-kube-api-access-pkjp4" (OuterVolumeSpecName: "kube-api-access-pkjp4") pod "88590f47-c61b-4e8b-b871-c479ccc641cf" (UID: "88590f47-c61b-4e8b-b871-c479ccc641cf"). InnerVolumeSpecName "kube-api-access-pkjp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:41:45.795924 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.795895    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-kube-api-access-tj75m" (OuterVolumeSpecName: "kube-api-access-tj75m") pod "2e2f74f1-d75b-44fa-9ebb-12c0b566796b" (UID: "2e2f74f1-d75b-44fa-9ebb-12c0b566796b"). InnerVolumeSpecName "kube-api-access-tj75m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:41:45.800058 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.800037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"]
Sep 16 18:41:45.894618 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.894583    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.894803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.894704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7675640-fd5f-49d1-9711-c4deb19aaa88-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.894803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.894737    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2s9m\" (UniqueName: \"kubernetes.io/projected/d7675640-fd5f-49d1-9711-c4deb19aaa88-kube-api-access-g2s9m\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.894897 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.894846    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.894963 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.894919    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkjp4\" (UniqueName: \"kubernetes.io/projected/88590f47-c61b-4e8b-b871-c479ccc641cf-kube-api-access-pkjp4\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:41:45.894963 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.894930    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88590f47-c61b-4e8b-b871-c479ccc641cf-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:41:45.894963 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.894946    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj75m\" (UniqueName: \"kubernetes.io/projected/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-kube-api-access-tj75m\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:41:45.894963 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.894959    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2e2f74f1-d75b-44fa-9ebb-12c0b566796b-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:41:45.995676 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.995644    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7675640-fd5f-49d1-9711-c4deb19aaa88-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.995676 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.995678    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2s9m\" (UniqueName: \"kubernetes.io/projected/d7675640-fd5f-49d1-9711-c4deb19aaa88-kube-api-access-g2s9m\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.995906 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.995739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.995906 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.995787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.995992 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:41:45.995915    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-serving-cert: secret "maas-api-e2e-shared-b-w2-520999-serving-cert" not found
Sep 16 18:41:45.995992 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:41:45.995982    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-maas-api-tls podName:d7675640-fd5f-49d1-9711-c4deb19aaa88 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:46.495963058 +0000 UTC m=+1530.052256357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-maas-api-tls") pod "maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8" (UID: "d7675640-fd5f-49d1-9711-c4deb19aaa88") : secret "maas-api-e2e-shared-b-w2-520999-serving-cert" not found
Sep 16 18:41:45.996312 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.996294    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7675640-fd5f-49d1-9711-c4deb19aaa88-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:45.998225 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:45.998192    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:46.012041 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.012015    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2s9m\" (UniqueName: \"kubernetes.io/projected/d7675640-fd5f-49d1-9711-c4deb19aaa88-kube-api-access-g2s9m\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:46.463541 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.463505    2576 generic.go:356] "Generic (PLEG): container finished" podID="88590f47-c61b-4e8b-b871-c479ccc641cf" containerID="05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027" exitCode=0
Sep 16 18:41:46.463992 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.463548    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"
Sep 16 18:41:46.463992 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.463591    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl" event={"ID":"88590f47-c61b-4e8b-b871-c479ccc641cf","Type":"ContainerDied","Data":"05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027"}
Sep 16 18:41:46.463992 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.463640    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl" event={"ID":"88590f47-c61b-4e8b-b871-c479ccc641cf","Type":"ContainerDied","Data":"3884e7c7998c722e1b33f3802f2f215d965ed971a08b2e106f0b1953f8e5ba5f"}
Sep 16 18:41:46.463992 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.463660    2576 scope.go:117] "RemoveContainer" containerID="05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027"
Sep 16 18:41:46.465018 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.464951    2576 generic.go:356] "Generic (PLEG): container finished" podID="2e2f74f1-d75b-44fa-9ebb-12c0b566796b" containerID="e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958" exitCode=0
Sep 16 18:41:46.465018 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.464980    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm" event={"ID":"2e2f74f1-d75b-44fa-9ebb-12c0b566796b","Type":"ContainerDied","Data":"e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958"}
Sep 16 18:41:46.465018 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.465009    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"
Sep 16 18:41:46.465146 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.465027    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm" event={"ID":"2e2f74f1-d75b-44fa-9ebb-12c0b566796b","Type":"ContainerDied","Data":"d28778ecd65346f9036cec393df75d591461bcb2e207fbdb43dd4afad872b1d0"}
Sep 16 18:41:46.478280 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.478258    2576 scope.go:117] "RemoveContainer" containerID="05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027"
Sep 16 18:41:46.478709 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:41:46.478668    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027\": container with ID starting with 05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027 not found: ID does not exist" containerID="05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027"
Sep 16 18:41:46.479801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.478712    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027"} err="failed to get container status \"05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027\": rpc error: code = NotFound desc = could not find container \"05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027\": container with ID starting with 05dd476b28353bae153bb989b2c0a86f805dbbb05c5512a786fc476de4f58027 not found: ID does not exist"
Sep 16 18:41:46.479801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.478736    2576 scope.go:117] "RemoveContainer" containerID="e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958"
Sep 16 18:41:46.488081 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.488065    2576 scope.go:117] "RemoveContainer" containerID="e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958"
Sep 16 18:41:46.488358 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:41:46.488337    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958\": container with ID starting with e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958 not found: ID does not exist" containerID="e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958"
Sep 16 18:41:46.488444 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.488362    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958"} err="failed to get container status \"e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958\": rpc error: code = NotFound desc = could not find container \"e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958\": container with ID starting with e8ee4864c3d25ba89588c351c90f8b61a8da3a3fff3d0088515185cbb252f958 not found: ID does not exist"
Sep 16 18:41:46.500640 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.500615    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:46.502794 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.502774    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:46.507888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.507866    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"]
Sep 16 18:41:46.513202 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.513180    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm"]
Sep 16 18:41:46.532525 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.532503    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"]
Sep 16 18:41:46.534550 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.534532    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl"]
Sep 16 18:41:46.683341 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.683317    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:46.945507 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.945478    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e2f74f1-d75b-44fa-9ebb-12c0b566796b" path="/var/lib/kubelet/pods/2e2f74f1-d75b-44fa-9ebb-12c0b566796b/volumes"
Sep 16 18:41:46.945868 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:46.945855    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88590f47-c61b-4e8b-b871-c479ccc641cf" path="/var/lib/kubelet/pods/88590f47-c61b-4e8b-b871-c479ccc641cf/volumes"
Sep 16 18:41:47.006045 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.006021    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"]
Sep 16 18:41:47.007336 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:41:47.007308    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7675640_fd5f_49d1_9711_c4deb19aaa88.slice/crio-b3805d9cf400c428f3ed6491fc38898d9b5915791d57f4cf3133175303958bde WatchSource:0}: Error finding container b3805d9cf400c428f3ed6491fc38898d9b5915791d57f4cf3133175303958bde: Status 404 returned error can't find the container with id b3805d9cf400c428f3ed6491fc38898d9b5915791d57f4cf3133175303958bde
Sep 16 18:41:47.008096 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.008074    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"]
Sep 16 18:41:47.013077 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.013060    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:47.028163 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:41:47.028134    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-e2e-shared-b-w2-520999-dockercfg-4jgwm\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-520999-dockercfg-4jgwm\"" type="*v1.Secret"
Sep 16 18:41:47.028268 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:41:47.028151    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:47.028268 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:41:47.028163    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"payload-processing-plugins-e2e-shared-b-w2-520999\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-520999\"" type="*v1.ConfigMap"
Sep 16 18:41:47.073489 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.073456    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"]
Sep 16 18:41:47.107594 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.107556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6q9q\" (UniqueName: \"kubernetes.io/projected/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-kube-api-access-c6q9q\") pod \"payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29\" (UID: \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:47.107726 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.107608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29\" (UID: \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:47.208380 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.208350    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6q9q\" (UniqueName: \"kubernetes.io/projected/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-kube-api-access-c6q9q\") pod \"payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29\" (UID: \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:47.208536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.208388    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29\" (UID: \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:47.223010 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.222980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6q9q\" (UniqueName: \"kubernetes.io/projected/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-kube-api-access-c6q9q\") pod \"payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29\" (UID: \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:47.473727 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.473689    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8" event={"ID":"d7675640-fd5f-49d1-9711-c4deb19aaa88","Type":"ContainerStarted","Data":"26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f"}
Sep 16 18:41:47.474195 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.473747    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8" event={"ID":"d7675640-fd5f-49d1-9711-c4deb19aaa88","Type":"ContainerStarted","Data":"b3805d9cf400c428f3ed6491fc38898d9b5915791d57f4cf3133175303958bde"}
Sep 16 18:41:47.474195 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.473822    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:41:47.525896 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:47.525846    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8" podStartSLOduration=2.1895569249999998 podStartE2EDuration="2.525828632s" totalImagesPullingTime="336.271707ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:47.523916684 +0000 UTC m=+1531.080209996" watchObservedRunningTime="2026-09-16 18:41:47.525828632 +0000 UTC m=+1531.082121943"
Sep 16 18:41:48.086062 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:48.086029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-520999-dockercfg-4jgwm\""
Sep 16 18:41:48.209256 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:41:48.209203    2576 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-shared-b-w2-520999: failed to sync configmap cache: timed out waiting for the condition
Sep 16 18:41:48.209430 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:41:48.209308    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-plugins-config-volume podName:9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:48.709290958 +0000 UTC m=+1532.265584248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-plugins-config-volume") pod "payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29" (UID: "9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4") : failed to sync configmap cache: timed out waiting for the condition
Sep 16 18:41:48.394349 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:48.394327    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-520999\""
Sep 16 18:41:48.725389 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:48.725303    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29\" (UID: \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:48.726006 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:48.725989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29\" (UID: \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:48.824056 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:48.824023    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:49.190766 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:49.190729    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"]
Sep 16 18:41:49.213317 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:41:49.213284    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9afa2b_a1ab_4ddb_8d7a_ccfd3feeada4.slice/crio-274823cb14c0fac8d10c4b1de1b541fd177eedd238f50cf8ca09a6e503783df0 WatchSource:0}: Error finding container 274823cb14c0fac8d10c4b1de1b541fd177eedd238f50cf8ca09a6e503783df0: Status 404 returned error can't find the container with id 274823cb14c0fac8d10c4b1de1b541fd177eedd238f50cf8ca09a6e503783df0
Sep 16 18:41:49.486615 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:49.486517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29" event={"ID":"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4","Type":"ContainerStarted","Data":"97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904"}
Sep 16 18:41:49.486615 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:49.486575    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29" event={"ID":"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4","Type":"ContainerStarted","Data":"274823cb14c0fac8d10c4b1de1b541fd177eedd238f50cf8ca09a6e503783df0"}
Sep 16 18:41:49.486839 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:49.486688    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:41:49.544180 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:49.544121    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29" podStartSLOduration=3.544104781 podStartE2EDuration="3.544104781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:49.542700617 +0000 UTC m=+1533.098993956" watchObservedRunningTime="2026-09-16 18:41:49.544104781 +0000 UTC m=+1533.100398091"
Sep 16 18:41:53.483930 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:41:53.483894    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:42:00.064958 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.064922    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7873-b0d918"]
Sep 16 18:42:00.069774 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.069752    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-b0d918"
Sep 16 18:42:00.072163 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.072137    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fd5lk\""
Sep 16 18:42:00.077250 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.077195    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7873-b0d918"]
Sep 16 18:42:00.143175 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.143138    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5kp6\" (UniqueName: \"kubernetes.io/projected/eee64dcd-d9c9-4107-80ae-053171a98077-kube-api-access-p5kp6\") pod \"test-curl-7873-b0d918\" (UID: \"eee64dcd-d9c9-4107-80ae-053171a98077\") " pod="opendatahub/test-curl-7873-b0d918"
Sep 16 18:42:00.243821 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.243781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5kp6\" (UniqueName: \"kubernetes.io/projected/eee64dcd-d9c9-4107-80ae-053171a98077-kube-api-access-p5kp6\") pod \"test-curl-7873-b0d918\" (UID: \"eee64dcd-d9c9-4107-80ae-053171a98077\") " pod="opendatahub/test-curl-7873-b0d918"
Sep 16 18:42:00.253431 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.253397    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5kp6\" (UniqueName: \"kubernetes.io/projected/eee64dcd-d9c9-4107-80ae-053171a98077-kube-api-access-p5kp6\") pod \"test-curl-7873-b0d918\" (UID: \"eee64dcd-d9c9-4107-80ae-053171a98077\") " pod="opendatahub/test-curl-7873-b0d918"
Sep 16 18:42:00.380463 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.380386    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-b0d918"
Sep 16 18:42:00.494325 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.494286    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:42:00.661187 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:00.661157    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7873-b0d918"]
Sep 16 18:42:00.664188 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:42:00.664162    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeee64dcd_d9c9_4107_80ae_053171a98077.slice/crio-12f415214241595d8f87cf63b2b2685b8e9f8158b493f57ff8b47aaad458f82f WatchSource:0}: Error finding container 12f415214241595d8f87cf63b2b2685b8e9f8158b493f57ff8b47aaad458f82f: Status 404 returned error can't find the container with id 12f415214241595d8f87cf63b2b2685b8e9f8158b493f57ff8b47aaad458f82f
Sep 16 18:42:01.556911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:01.556876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7873-b0d918" event={"ID":"eee64dcd-d9c9-4107-80ae-053171a98077","Type":"ContainerStarted","Data":"e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e"}
Sep 16 18:42:01.557351 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:01.556924    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7873-b0d918" event={"ID":"eee64dcd-d9c9-4107-80ae-053171a98077","Type":"ContainerStarted","Data":"12f415214241595d8f87cf63b2b2685b8e9f8158b493f57ff8b47aaad458f82f"}
Sep 16 18:42:02.332768 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.332673    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7873-b0d918" podStartSLOduration=2.332632005 podStartE2EDuration="2.332632005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:01.611666056 +0000 UTC m=+1545.167959368" watchObservedRunningTime="2026-09-16 18:42:02.332632005 +0000 UTC m=+1545.888925316"
Sep 16 18:42:02.334372 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.334349    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7873-b0d918"]
Sep 16 18:42:02.341377 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.341349    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7873-b0d918"]
Sep 16 18:42:02.564328 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:42:02.564296    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7873-b0d918" container="test-curl-7873-b0d918"
Sep 16 18:42:02.574805 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.574783    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"]
Sep 16 18:42:02.575379 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.575357    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eee64dcd-d9c9-4107-80ae-053171a98077" containerName="test-curl-7873-b0d918"
Sep 16 18:42:02.575379 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.575376    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eee64dcd-d9c9-4107-80ae-053171a98077" containerName="test-curl-7873-b0d918"
Sep 16 18:42:02.575556 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.575522    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eee64dcd-d9c9-4107-80ae-053171a98077" containerName="test-curl-7873-b0d918"
Sep 16 18:42:02.578988 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.578967    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.581938 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.581920    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-8edc30-serving-cert\""
Sep 16 18:42:02.592545 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.592521    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"]
Sep 16 18:42:02.668350 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.668320    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.668490 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.668418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.668490 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.668439    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.668569 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.668490    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8vsk\" (UniqueName: \"kubernetes.io/projected/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-kube-api-access-q8vsk\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.769432 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.769409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.769575 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.769479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.769575 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.769500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.769575 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.769532    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8vsk\" (UniqueName: \"kubernetes.io/projected/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-kube-api-access-q8vsk\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.770201 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.770180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.771902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.771879    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.772045 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.772025    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.778516 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.778487    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8vsk\" (UniqueName: \"kubernetes.io/projected/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-kube-api-access-q8vsk\") pod \"maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:02.890803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:02.890741    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:03.191993 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.191965    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"]
Sep 16 18:42:03.193198 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:42:03.193130    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ee0dd6_a874_4824_ba80_c2565efc8aa8.slice/crio-2b72f29f30fc4d533d051333131022bbc77e9e48501290baba0de2419a2055b2 WatchSource:0}: Error finding container 2b72f29f30fc4d533d051333131022bbc77e9e48501290baba0de2419a2055b2: Status 404 returned error can't find the container with id 2b72f29f30fc4d533d051333131022bbc77e9e48501290baba0de2419a2055b2
Sep 16 18:42:03.483661 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.483583    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"]
Sep 16 18:42:03.488503 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.488488    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:03.491286 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.491261    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-8edc30\""
Sep 16 18:42:03.492571 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.492552    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-8edc30-dockercfg-cwdwf\""
Sep 16 18:42:03.506159 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.506137    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"]
Sep 16 18:42:03.546271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.546244    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"]
Sep 16 18:42:03.550151 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.550132    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:03.572241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.572200    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn" event={"ID":"b1ee0dd6-a874-4824-ba80-c2565efc8aa8","Type":"ContainerStarted","Data":"2b72f29f30fc4d533d051333131022bbc77e9e48501290baba0de2419a2055b2"}
Sep 16 18:42:03.572573 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.572329    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7873-b0d918" podUID="eee64dcd-d9c9-4107-80ae-053171a98077" containerName="test-curl-7873-b0d918" containerID="cri-o://e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e" gracePeriod=2
Sep 16 18:42:03.578134 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.578109    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b52gr\" (UniqueName: \"kubernetes.io/projected/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-kube-api-access-b52gr\") pod \"payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z\" (UID: \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:03.578271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.578159    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/352524ef-3b81-4646-b685-7e3d535d6b66-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl\" (UID: \"352524ef-3b81-4646-b685-7e3d535d6b66\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:03.578271 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.578188    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkb5w\" (UniqueName: \"kubernetes.io/projected/352524ef-3b81-4646-b685-7e3d535d6b66-kube-api-access-gkb5w\") pod \"payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl\" (UID: \"352524ef-3b81-4646-b685-7e3d535d6b66\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:03.578376 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.578267    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z\" (UID: \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:03.630803 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.630782    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"]
Sep 16 18:42:03.679811 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.679784    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b52gr\" (UniqueName: \"kubernetes.io/projected/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-kube-api-access-b52gr\") pod \"payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z\" (UID: \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:03.679917 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.679831    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/352524ef-3b81-4646-b685-7e3d535d6b66-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl\" (UID: \"352524ef-3b81-4646-b685-7e3d535d6b66\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:03.679917 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.679874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkb5w\" (UniqueName: \"kubernetes.io/projected/352524ef-3b81-4646-b685-7e3d535d6b66-kube-api-access-gkb5w\") pod \"payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl\" (UID: \"352524ef-3b81-4646-b685-7e3d535d6b66\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:03.679917 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.679909    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z\" (UID: \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:03.680596 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.680577    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z\" (UID: \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:03.680648 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.680631    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/352524ef-3b81-4646-b685-7e3d535d6b66-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl\" (UID: \"352524ef-3b81-4646-b685-7e3d535d6b66\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:03.691910 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.691887    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b52gr\" (UniqueName: \"kubernetes.io/projected/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-kube-api-access-b52gr\") pod \"payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z\" (UID: \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:03.694008 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.693979    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkb5w\" (UniqueName: \"kubernetes.io/projected/352524ef-3b81-4646-b685-7e3d535d6b66-kube-api-access-gkb5w\") pod \"payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl\" (UID: \"352524ef-3b81-4646-b685-7e3d535d6b66\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:03.799349 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.799274    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:03.860412 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:03.860383    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:04.147734 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.147711    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"]
Sep 16 18:42:04.149566 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:42:04.149540    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a5a3d9c_6fce_4b7a_9593_ec822cee76b8.slice/crio-780485d23672af97641d807b9d9f911ba9c2d1db1df8894311978dbab1e9a9a5 WatchSource:0}: Error finding container 780485d23672af97641d807b9d9f911ba9c2d1db1df8894311978dbab1e9a9a5: Status 404 returned error can't find the container with id 780485d23672af97641d807b9d9f911ba9c2d1db1df8894311978dbab1e9a9a5
Sep 16 18:42:04.206027 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.206002    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"]
Sep 16 18:42:04.207529 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:42:04.207496    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod352524ef_3b81_4646_b685_7e3d535d6b66.slice/crio-dbcf2564b3ca7bd17a9711e06e8256c48f00a39fa1ddc568eb0aec3d5b529695 WatchSource:0}: Error finding container dbcf2564b3ca7bd17a9711e06e8256c48f00a39fa1ddc568eb0aec3d5b529695: Status 404 returned error can't find the container with id dbcf2564b3ca7bd17a9711e06e8256c48f00a39fa1ddc568eb0aec3d5b529695
Sep 16 18:42:04.580996 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.580905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z" event={"ID":"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8","Type":"ContainerStarted","Data":"96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8"}
Sep 16 18:42:04.580996 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.580987    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z" event={"ID":"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8","Type":"ContainerStarted","Data":"780485d23672af97641d807b9d9f911ba9c2d1db1df8894311978dbab1e9a9a5"}
Sep 16 18:42:04.581504 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.581017    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:04.582663 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.582635    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn" event={"ID":"b1ee0dd6-a874-4824-ba80-c2565efc8aa8","Type":"ContainerStarted","Data":"853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0"}
Sep 16 18:42:04.582804 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.582689    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:04.584265 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.584245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl" event={"ID":"352524ef-3b81-4646-b685-7e3d535d6b66","Type":"ContainerStarted","Data":"cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606"}
Sep 16 18:42:04.584367 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.584279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl" event={"ID":"352524ef-3b81-4646-b685-7e3d535d6b66","Type":"ContainerStarted","Data":"dbcf2564b3ca7bd17a9711e06e8256c48f00a39fa1ddc568eb0aec3d5b529695"}
Sep 16 18:42:04.584367 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.584318    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:04.699167 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.699108    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z" podStartSLOduration=1.699094927 podStartE2EDuration="1.699094927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:04.695868285 +0000 UTC m=+1548.252161586" watchObservedRunningTime="2026-09-16 18:42:04.699094927 +0000 UTC m=+1548.255388237"
Sep 16 18:42:04.840599 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.840513    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn" podStartSLOduration=2.430606886 podStartE2EDuration="2.840497782s" totalImagesPullingTime="409.890896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:04.839234744 +0000 UTC m=+1548.395528051" watchObservedRunningTime="2026-09-16 18:42:04.840497782 +0000 UTC m=+1548.396791093"
Sep 16 18:42:04.840713 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:04.840640    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl" podStartSLOduration=1.840635442 podStartE2EDuration="1.840635442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:04.726054444 +0000 UTC m=+1548.282347754" watchObservedRunningTime="2026-09-16 18:42:04.840635442 +0000 UTC m=+1548.396928794"
Sep 16 18:42:05.759516 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:05.759493    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-b0d918"
Sep 16 18:42:05.804003 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:05.803979    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5kp6\" (UniqueName: \"kubernetes.io/projected/eee64dcd-d9c9-4107-80ae-053171a98077-kube-api-access-p5kp6\") pod \"eee64dcd-d9c9-4107-80ae-053171a98077\" (UID: \"eee64dcd-d9c9-4107-80ae-053171a98077\") "
Sep 16 18:42:05.806023 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:05.805995    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eee64dcd-d9c9-4107-80ae-053171a98077-kube-api-access-p5kp6" (OuterVolumeSpecName: "kube-api-access-p5kp6") pod "eee64dcd-d9c9-4107-80ae-053171a98077" (UID: "eee64dcd-d9c9-4107-80ae-053171a98077"). InnerVolumeSpecName "kube-api-access-p5kp6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:42:05.904740 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:05.904714    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5kp6\" (UniqueName: \"kubernetes.io/projected/eee64dcd-d9c9-4107-80ae-053171a98077-kube-api-access-p5kp6\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:42:06.597783 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:06.597754    2576 generic.go:356] "Generic (PLEG): container finished" podID="eee64dcd-d9c9-4107-80ae-053171a98077" containerID="e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e" exitCode=137
Sep 16 18:42:06.597955 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:06.597796    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-b0d918"
Sep 16 18:42:06.597955 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:06.597859    2576 scope.go:117] "RemoveContainer" containerID="e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e"
Sep 16 18:42:06.607639 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:06.607621    2576 scope.go:117] "RemoveContainer" containerID="e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e"
Sep 16 18:42:06.607932 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:42:06.607908    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e\": container with ID starting with e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e not found: ID does not exist" containerID="e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e"
Sep 16 18:42:06.608035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:06.607942    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e"} err="failed to get container status \"e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e\": rpc error: code = NotFound desc = could not find container \"e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e\": container with ID starting with e72300969b9ae18ce2f10ad592b57270da4c961648201040dbc863e8b3548d4e not found: ID does not exist"
Sep 16 18:42:06.609042 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:42:06.609018    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7873-b0d918\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="eee64dcd-d9c9-4107-80ae-053171a98077" pod="opendatahub/test-curl-7873-b0d918"
Sep 16 18:42:06.944492 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:42:06.944458    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7873-b0d918\" is forbidden: User \"system:node:ip-10-0-129-142.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-129-142.ec2.internal' and this object" podUID="eee64dcd-d9c9-4107-80ae-053171a98077" pod="opendatahub/test-curl-7873-b0d918"
Sep 16 18:42:06.945104 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:06.945080    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eee64dcd-d9c9-4107-80ae-053171a98077" path="/var/lib/kubelet/pods/eee64dcd-d9c9-4107-80ae-053171a98077/volumes"
Sep 16 18:42:10.596617 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:10.596590    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:42:13.877909 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:13.877875    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"]
Sep 16 18:42:13.882720 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:13.882705    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:13.885090 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:13.885070    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ab83f76b\""
Sep 16 18:42:13.885478 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:13.885462    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ab83f76b-dockercfg-r5qz7\""
Sep 16 18:42:13.898155 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:13.898132    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"]
Sep 16 18:42:13.983035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:13.983009    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg\" (UID: \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:13.983196 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:13.983060    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftktr\" (UniqueName: \"kubernetes.io/projected/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-kube-api-access-ftktr\") pod \"payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg\" (UID: \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:14.084693 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.084662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg\" (UID: \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:14.084857 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.084718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftktr\" (UniqueName: \"kubernetes.io/projected/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-kube-api-access-ftktr\") pod \"payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg\" (UID: \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:14.085355 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.085334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg\" (UID: \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:14.093460 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.093431    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftktr\" (UniqueName: \"kubernetes.io/projected/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-kube-api-access-ftktr\") pod \"payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg\" (UID: \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:14.193265 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.193230    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:14.489729 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.489633    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"]
Sep 16 18:42:14.492519 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:42:14.492492    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod055da7ce_33bc_4ad9_a625_0d9eb6bd15b6.slice/crio-9e2428c1c97e23fdb8900a85ea6fe83d06fd2d5853d5868cd05bd8e870024c86 WatchSource:0}: Error finding container 9e2428c1c97e23fdb8900a85ea6fe83d06fd2d5853d5868cd05bd8e870024c86: Status 404 returned error can't find the container with id 9e2428c1c97e23fdb8900a85ea6fe83d06fd2d5853d5868cd05bd8e870024c86
Sep 16 18:42:14.655787 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.655754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg" event={"ID":"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6","Type":"ContainerStarted","Data":"fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931"}
Sep 16 18:42:14.655970 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.655810    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg" event={"ID":"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6","Type":"ContainerStarted","Data":"9e2428c1c97e23fdb8900a85ea6fe83d06fd2d5853d5868cd05bd8e870024c86"}
Sep 16 18:42:14.655970 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.655863    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:14.690659 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:14.690613    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg" podStartSLOduration=1.690598595 podStartE2EDuration="1.690598595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:14.685804017 +0000 UTC m=+1558.242097319" watchObservedRunningTime="2026-09-16 18:42:14.690598595 +0000 UTC m=+1558.246891906"
Sep 16 18:42:15.593429 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:15.593399    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:42:15.593868 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:15.593501    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:42:17.151281 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:17.151251    2576 scope.go:117] "RemoveContainer" containerID="1fe60f0f4f0679cfc63f70b23a895ba2a4de12e7845592fd6b6f163e531949b0"
Sep 16 18:42:23.875530 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:23.875501    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"]
Sep 16 18:42:23.879584 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:23.879562    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:23.881972 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:23.881947    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-ee3af0-serving-cert\""
Sep 16 18:42:23.891243 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:23.891220    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"]
Sep 16 18:42:23.982470 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:23.982436    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68a9f3b7-9a28-4795-b09f-2f19a402bd52-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:23.982629 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:23.982479    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:23.982629 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:23.982503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p5s2\" (UniqueName: \"kubernetes.io/projected/68a9f3b7-9a28-4795-b09f-2f19a402bd52-kube-api-access-7p5s2\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:23.982752 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:23.982662    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.083958 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.083933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.084297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.084270    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68a9f3b7-9a28-4795-b09f-2f19a402bd52-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.084430 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.084344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.084430 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.084381    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p5s2\" (UniqueName: \"kubernetes.io/projected/68a9f3b7-9a28-4795-b09f-2f19a402bd52-kube-api-access-7p5s2\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.084541 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:42:24.084465    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-serving-cert: secret "maas-api-e2e-shared-b-w3-ee3af0-serving-cert" not found
Sep 16 18:42:24.084605 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:42:24.084557    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-maas-api-tls podName:68a9f3b7-9a28-4795-b09f-2f19a402bd52 nodeName:}" failed. No retries permitted until 2026-09-16 18:42:24.584535254 +0000 UTC m=+1568.140828554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-maas-api-tls") pod "maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq" (UID: "68a9f3b7-9a28-4795-b09f-2f19a402bd52") : secret "maas-api-e2e-shared-b-w3-ee3af0-serving-cert" not found
Sep 16 18:42:24.084918 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.084901    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68a9f3b7-9a28-4795-b09f-2f19a402bd52-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.086439 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.086419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.104826 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.104797    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p5s2\" (UniqueName: \"kubernetes.io/projected/68a9f3b7-9a28-4795-b09f-2f19a402bd52-kube-api-access-7p5s2\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.590504 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.590470    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.593065 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.593034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.794590 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.794562    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:24.881692 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.881659    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"]
Sep 16 18:42:24.886613 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.886592    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:42:24.891194 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.891151    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-ee3af0\""
Sep 16 18:42:24.891332 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.891255    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-ee3af0-dockercfg-msgrm\""
Sep 16 18:42:24.897633 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.897612    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"]
Sep 16 18:42:24.993618 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.993585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr\" (UID: \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:42:24.993794 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:24.993700    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tstxl\" (UniqueName: \"kubernetes.io/projected/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-kube-api-access-tstxl\") pod \"payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr\" (UID: \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:42:25.080057 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.080032    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"]
Sep 16 18:42:25.081385 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:42:25.081361    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a9f3b7_9a28_4795_b09f_2f19a402bd52.slice/crio-206c502722ddd74e40c3c6c0bb03b373604fe6091efa5c8331e93e5bc5832988 WatchSource:0}: Error finding container 206c502722ddd74e40c3c6c0bb03b373604fe6091efa5c8331e93e5bc5832988: Status 404 returned error can't find the container with id 206c502722ddd74e40c3c6c0bb03b373604fe6091efa5c8331e93e5bc5832988
Sep 16 18:42:25.095231 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.095137    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr\" (UID: \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:42:25.095341 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.095255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tstxl\" (UniqueName: \"kubernetes.io/projected/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-kube-api-access-tstxl\") pod \"payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr\" (UID: \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:42:25.095954 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.095936    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr\" (UID: \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:42:25.105127 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.105106    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tstxl\" (UniqueName: \"kubernetes.io/projected/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-kube-api-access-tstxl\") pod \"payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr\" (UID: \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:42:25.200665 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.200626    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:42:25.487415 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.487394    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"]
Sep 16 18:42:25.488779 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:42:25.488759    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e226d70_ee7b_4b2b_a348_68bb0d68c2ad.slice/crio-c77d24784dee9448606eb93269cc08cb98c66a9e10b725e73de13effc00c6a84 WatchSource:0}: Error finding container c77d24784dee9448606eb93269cc08cb98c66a9e10b725e73de13effc00c6a84: Status 404 returned error can't find the container with id c77d24784dee9448606eb93269cc08cb98c66a9e10b725e73de13effc00c6a84
Sep 16 18:42:25.664975 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.664943    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:42:25.721677 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.721578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq" event={"ID":"68a9f3b7-9a28-4795-b09f-2f19a402bd52","Type":"ContainerStarted","Data":"346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504"}
Sep 16 18:42:25.721677 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.721662    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq" event={"ID":"68a9f3b7-9a28-4795-b09f-2f19a402bd52","Type":"ContainerStarted","Data":"206c502722ddd74e40c3c6c0bb03b373604fe6091efa5c8331e93e5bc5832988"}
Sep 16 18:42:25.721898 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.721750    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:25.723732 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.723704    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr" event={"ID":"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad","Type":"ContainerStarted","Data":"a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9"}
Sep 16 18:42:25.723887 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.723754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr" event={"ID":"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad","Type":"ContainerStarted","Data":"c77d24784dee9448606eb93269cc08cb98c66a9e10b725e73de13effc00c6a84"}
Sep 16 18:42:25.723887 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.723786    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:42:25.757557 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.757508    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq" podStartSLOduration=2.415476069 podStartE2EDuration="2.75749311s" totalImagesPullingTime="342.017041ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:25.752602355 +0000 UTC m=+1569.308895667" watchObservedRunningTime="2026-09-16 18:42:25.75749311 +0000 UTC m=+1569.313786439"
Sep 16 18:42:25.786412 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:25.786368    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr" podStartSLOduration=1.786353927 podStartE2EDuration="1.786353927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:25.782923887 +0000 UTC m=+1569.339217197" watchObservedRunningTime="2026-09-16 18:42:25.786353927 +0000 UTC m=+1569.342647295"
Sep 16 18:42:31.735620 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:31.735586    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:42:36.732027 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:42:36.731985    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:43:17.187296 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:43:17.187229    2576 scope.go:117] "RemoveContainer" containerID="351a3bf9327b23917625fa1020b4ed9949a1ff0d0f8f1d01d1f0425a1204f74d"
Sep 16 18:44:18.951708 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:18.951675    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"]
Sep 16 18:44:29.832595 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:29.832560    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"]
Sep 16 18:44:29.832980 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:29.832820    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl" podUID="9b39d210-6738-4cf7-82e5-0f8025c69759" containerName="payload-processing" containerID="cri-o://98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b" gracePeriod=30
Sep 16 18:44:30.098605 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.098542    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:44:30.236560 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.236531    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtqlt\" (UniqueName: \"kubernetes.io/projected/9b39d210-6738-4cf7-82e5-0f8025c69759-kube-api-access-jtqlt\") pod \"9b39d210-6738-4cf7-82e5-0f8025c69759\" (UID: \"9b39d210-6738-4cf7-82e5-0f8025c69759\") "
Sep 16 18:44:30.236716 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.236626    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b39d210-6738-4cf7-82e5-0f8025c69759-plugins-config-volume\") pod \"9b39d210-6738-4cf7-82e5-0f8025c69759\" (UID: \"9b39d210-6738-4cf7-82e5-0f8025c69759\") "
Sep 16 18:44:30.237022 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.236997    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b39d210-6738-4cf7-82e5-0f8025c69759-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b39d210-6738-4cf7-82e5-0f8025c69759" (UID: "9b39d210-6738-4cf7-82e5-0f8025c69759"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:44:30.238625 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.238595    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b39d210-6738-4cf7-82e5-0f8025c69759-kube-api-access-jtqlt" (OuterVolumeSpecName: "kube-api-access-jtqlt") pod "9b39d210-6738-4cf7-82e5-0f8025c69759" (UID: "9b39d210-6738-4cf7-82e5-0f8025c69759"). InnerVolumeSpecName "kube-api-access-jtqlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:44:30.337773 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.337743    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtqlt\" (UniqueName: \"kubernetes.io/projected/9b39d210-6738-4cf7-82e5-0f8025c69759-kube-api-access-jtqlt\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:44:30.337773 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.337770    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b39d210-6738-4cf7-82e5-0f8025c69759-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:44:30.417525 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.417489    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b39d210-6738-4cf7-82e5-0f8025c69759" containerID="98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b" exitCode=0
Sep 16 18:44:30.417710 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.417546    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"
Sep 16 18:44:30.417710 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.417584    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl" event={"ID":"9b39d210-6738-4cf7-82e5-0f8025c69759","Type":"ContainerDied","Data":"98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b"}
Sep 16 18:44:30.417710 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.417651    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl" event={"ID":"9b39d210-6738-4cf7-82e5-0f8025c69759","Type":"ContainerDied","Data":"6576be4febff2908004ffd25a4a5c2505716757839d9a6454f0a91b91c301c8c"}
Sep 16 18:44:30.417710 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.417682    2576 scope.go:117] "RemoveContainer" containerID="98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b"
Sep 16 18:44:30.427779 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.427763    2576 scope.go:117] "RemoveContainer" containerID="98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b"
Sep 16 18:44:30.428038 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:44:30.428023    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b\": container with ID starting with 98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b not found: ID does not exist" containerID="98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b"
Sep 16 18:44:30.428091 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.428043    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b"} err="failed to get container status \"98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b\": rpc error: code = NotFound desc = could not find container \"98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b\": container with ID starting with 98736c5ecd3cd5abda655d3d4269e4ef563b1f99d776d3480513e48e57b0b34b not found: ID does not exist"
Sep 16 18:44:30.440520 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.440494    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"]
Sep 16 18:44:30.445297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.445273    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl"]
Sep 16 18:44:30.945490 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:30.945453    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b39d210-6738-4cf7-82e5-0f8025c69759" path="/var/lib/kubelet/pods/9b39d210-6738-4cf7-82e5-0f8025c69759/volumes"
Sep 16 18:44:33.971122 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:33.971074    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg" podUID="e48e7cb1-245c-4da0-9650-13188bcc50a0" containerName="main" containerID="cri-o://7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9" gracePeriod=45
Sep 16 18:44:34.224184 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.224123    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:44:34.377786 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.377756    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-dshm\") pod \"e48e7cb1-245c-4da0-9650-13188bcc50a0\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") "
Sep 16 18:44:34.377980 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.377810    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-tmp-dir\") pod \"e48e7cb1-245c-4da0-9650-13188bcc50a0\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") "
Sep 16 18:44:34.377980 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.377857    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-home\") pod \"e48e7cb1-245c-4da0-9650-13188bcc50a0\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") "
Sep 16 18:44:34.377980 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.377888    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-model-cache\") pod \"e48e7cb1-245c-4da0-9650-13188bcc50a0\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") "
Sep 16 18:44:34.378169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.377983    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tcs27\" (UniqueName: \"kubernetes.io/projected/e48e7cb1-245c-4da0-9650-13188bcc50a0-kube-api-access-tcs27\") pod \"e48e7cb1-245c-4da0-9650-13188bcc50a0\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") "
Sep 16 18:44:34.378169 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.378016    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e48e7cb1-245c-4da0-9650-13188bcc50a0-tls-certs\") pod \"e48e7cb1-245c-4da0-9650-13188bcc50a0\" (UID: \"e48e7cb1-245c-4da0-9650-13188bcc50a0\") "
Sep 16 18:44:34.378418 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.378372    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-model-cache" (OuterVolumeSpecName: "model-cache") pod "e48e7cb1-245c-4da0-9650-13188bcc50a0" (UID: "e48e7cb1-245c-4da0-9650-13188bcc50a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:44:34.378418 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.378369    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-home" (OuterVolumeSpecName: "home") pod "e48e7cb1-245c-4da0-9650-13188bcc50a0" (UID: "e48e7cb1-245c-4da0-9650-13188bcc50a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:44:34.378613 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.378407    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e48e7cb1-245c-4da0-9650-13188bcc50a0" (UID: "e48e7cb1-245c-4da0-9650-13188bcc50a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:44:34.381138 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.381100    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e48e7cb1-245c-4da0-9650-13188bcc50a0-kube-api-access-tcs27" (OuterVolumeSpecName: "kube-api-access-tcs27") pod "e48e7cb1-245c-4da0-9650-13188bcc50a0" (UID: "e48e7cb1-245c-4da0-9650-13188bcc50a0"). InnerVolumeSpecName "kube-api-access-tcs27". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:44:34.381709 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.381656    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-dshm" (OuterVolumeSpecName: "dshm") pod "e48e7cb1-245c-4da0-9650-13188bcc50a0" (UID: "e48e7cb1-245c-4da0-9650-13188bcc50a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 18:44:34.385066 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.385045    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e48e7cb1-245c-4da0-9650-13188bcc50a0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e48e7cb1-245c-4da0-9650-13188bcc50a0" (UID: "e48e7cb1-245c-4da0-9650-13188bcc50a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:44:34.443328 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.443283    2576 generic.go:356] "Generic (PLEG): container finished" podID="e48e7cb1-245c-4da0-9650-13188bcc50a0" containerID="7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9" exitCode=0
Sep 16 18:44:34.443328 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.443329    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"
Sep 16 18:44:34.443547 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.443344    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg" event={"ID":"e48e7cb1-245c-4da0-9650-13188bcc50a0","Type":"ContainerDied","Data":"7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9"}
Sep 16 18:44:34.443547 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.443376    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg" event={"ID":"e48e7cb1-245c-4da0-9650-13188bcc50a0","Type":"ContainerDied","Data":"09114d439c087b8175ce7e8a69f39b0c8ac54be0103a69de5ec32be58c71764c"}
Sep 16 18:44:34.443547 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.443392    2576 scope.go:117] "RemoveContainer" containerID="7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9"
Sep 16 18:44:34.452591 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.452572    2576 scope.go:117] "RemoveContainer" containerID="7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9"
Sep 16 18:44:34.452881 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:44:34.452861    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9\": container with ID starting with 7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9 not found: ID does not exist" containerID="7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9"
Sep 16 18:44:34.452940 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.452887    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9"} err="failed to get container status \"7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9\": rpc error: code = NotFound desc = could not find container \"7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9\": container with ID starting with 7144baac1ec0838f4f2a2a526ce512639ca774575207ca389067218a4c8fc8c9 not found: ID does not exist"
Sep 16 18:44:34.464350 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.464309    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"]
Sep 16 18:44:34.467053 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.467033    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-909a40/distinct-w0-909a40-kserve-6d9c8f9cb-zdnmg"]
Sep 16 18:44:34.478911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.478862    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-dshm\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:44:34.478911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.478882    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-tmp-dir\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:44:34.478911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.478890    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-home\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:44:34.478911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.478897    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e48e7cb1-245c-4da0-9650-13188bcc50a0-model-cache\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:44:34.478911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.478905    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcs27\" (UniqueName: \"kubernetes.io/projected/e48e7cb1-245c-4da0-9650-13188bcc50a0-kube-api-access-tcs27\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:44:34.478911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.478913    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e48e7cb1-245c-4da0-9650-13188bcc50a0-tls-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:44:34.945396 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:44:34.945368    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e48e7cb1-245c-4da0-9650-13188bcc50a0" path="/var/lib/kubelet/pods/e48e7cb1-245c-4da0-9650-13188bcc50a0/volumes"
Sep 16 18:45:47.732175 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:47.732137    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"]
Sep 16 18:45:47.732963 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:47.732904    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg" podUID="055da7ce-33bc-4ad9-a625-0d9eb6bd15b6" containerName="payload-pre-processing" containerID="cri-o://fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931" gracePeriod=30
Sep 16 18:45:47.986980 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:47.986918    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:45:48.059658 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.059629    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftktr\" (UniqueName: \"kubernetes.io/projected/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-kube-api-access-ftktr\") pod \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\" (UID: \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\") "
Sep 16 18:45:48.059842 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.059730    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-plugins-config-volume\") pod \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\" (UID: \"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6\") "
Sep 16 18:45:48.060257 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.060108    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "055da7ce-33bc-4ad9-a625-0d9eb6bd15b6" (UID: "055da7ce-33bc-4ad9-a625-0d9eb6bd15b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:45:48.060662 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.060639    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:48.062003 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.061978    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-kube-api-access-ftktr" (OuterVolumeSpecName: "kube-api-access-ftktr") pod "055da7ce-33bc-4ad9-a625-0d9eb6bd15b6" (UID: "055da7ce-33bc-4ad9-a625-0d9eb6bd15b6"). InnerVolumeSpecName "kube-api-access-ftktr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:48.161620 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.161590    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftktr\" (UniqueName: \"kubernetes.io/projected/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6-kube-api-access-ftktr\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:48.856077 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.856038    2576 generic.go:356] "Generic (PLEG): container finished" podID="055da7ce-33bc-4ad9-a625-0d9eb6bd15b6" containerID="fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931" exitCode=0
Sep 16 18:45:48.856539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.856095    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"
Sep 16 18:45:48.856539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.856115    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg" event={"ID":"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6","Type":"ContainerDied","Data":"fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931"}
Sep 16 18:45:48.856539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.856174    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg" event={"ID":"055da7ce-33bc-4ad9-a625-0d9eb6bd15b6","Type":"ContainerDied","Data":"9e2428c1c97e23fdb8900a85ea6fe83d06fd2d5853d5868cd05bd8e870024c86"}
Sep 16 18:45:48.856539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.856234    2576 scope.go:117] "RemoveContainer" containerID="fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931"
Sep 16 18:45:48.865887 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.865869    2576 scope.go:117] "RemoveContainer" containerID="fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931"
Sep 16 18:45:48.866151 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:45:48.866132    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931\": container with ID starting with fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931 not found: ID does not exist" containerID="fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931"
Sep 16 18:45:48.866226 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.866164    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931"} err="failed to get container status \"fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931\": rpc error: code = NotFound desc = could not find container \"fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931\": container with ID starting with fe51df3b9120dae5bf4e871f5359eae010179380bd9a9b96a89e7644586d9931 not found: ID does not exist"
Sep 16 18:45:48.910085 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.910058    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"]
Sep 16 18:45:48.934721 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.934696    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg"]
Sep 16 18:45:48.946050 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:48.946025    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="055da7ce-33bc-4ad9-a625-0d9eb6bd15b6" path="/var/lib/kubelet/pods/055da7ce-33bc-4ad9-a625-0d9eb6bd15b6/volumes"
Sep 16 18:45:53.808395 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:53.808356    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"]
Sep 16 18:45:53.808912 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:53.808581    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn" podUID="b1ee0dd6-a874-4824-ba80-c2565efc8aa8" containerName="maas-api" containerID="cri-o://853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0" gracePeriod=30
Sep 16 18:45:53.869083 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:53.869055    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"]
Sep 16 18:45:53.869358 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:53.869318    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl" podUID="352524ef-3b81-4646-b685-7e3d535d6b66" containerName="payload-processing" containerID="cri-o://cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606" gracePeriod=30
Sep 16 18:45:53.878639 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:53.878531    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"]
Sep 16 18:45:53.879038 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:53.878928    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z" podUID="1a5a3d9c-6fce-4b7a-9593-ec822cee76b8" containerName="payload-pre-processing" containerID="cri-o://96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8" gracePeriod=30
Sep 16 18:45:54.146336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.146309    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:45:54.166104 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.166078    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:45:54.182679 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.182656    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:45:54.223033 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223004    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-openshift-service-ca\") pod \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") "
Sep 16 18:45:54.223202 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223053    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-metrics-certs\") pod \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") "
Sep 16 18:45:54.223202 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223146    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/352524ef-3b81-4646-b685-7e3d535d6b66-plugins-config-volume\") pod \"352524ef-3b81-4646-b685-7e3d535d6b66\" (UID: \"352524ef-3b81-4646-b685-7e3d535d6b66\") "
Sep 16 18:45:54.223202 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223173    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkb5w\" (UniqueName: \"kubernetes.io/projected/352524ef-3b81-4646-b685-7e3d535d6b66-kube-api-access-gkb5w\") pod \"352524ef-3b81-4646-b685-7e3d535d6b66\" (UID: \"352524ef-3b81-4646-b685-7e3d535d6b66\") "
Sep 16 18:45:54.223202 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223194    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-maas-api-tls\") pod \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") "
Sep 16 18:45:54.223457 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223268    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8vsk\" (UniqueName: \"kubernetes.io/projected/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-kube-api-access-q8vsk\") pod \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\" (UID: \"b1ee0dd6-a874-4824-ba80-c2565efc8aa8\") "
Sep 16 18:45:54.223515 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223466    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1ee0dd6-a874-4824-ba80-c2565efc8aa8" (UID: "b1ee0dd6-a874-4824-ba80-c2565efc8aa8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:45:54.223565 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223544    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/352524ef-3b81-4646-b685-7e3d535d6b66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "352524ef-3b81-4646-b685-7e3d535d6b66" (UID: "352524ef-3b81-4646-b685-7e3d535d6b66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:45:54.223668 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223650    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-openshift-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:54.223726 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.223677    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/352524ef-3b81-4646-b685-7e3d535d6b66-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:54.225371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.225349    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1ee0dd6-a874-4824-ba80-c2565efc8aa8" (UID: "b1ee0dd6-a874-4824-ba80-c2565efc8aa8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:45:54.225493 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.225443    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-kube-api-access-q8vsk" (OuterVolumeSpecName: "kube-api-access-q8vsk") pod "b1ee0dd6-a874-4824-ba80-c2565efc8aa8" (UID: "b1ee0dd6-a874-4824-ba80-c2565efc8aa8"). InnerVolumeSpecName "kube-api-access-q8vsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:54.225721 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.225696    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b1ee0dd6-a874-4824-ba80-c2565efc8aa8" (UID: "b1ee0dd6-a874-4824-ba80-c2565efc8aa8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:45:54.225721 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.225709    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/352524ef-3b81-4646-b685-7e3d535d6b66-kube-api-access-gkb5w" (OuterVolumeSpecName: "kube-api-access-gkb5w") pod "352524ef-3b81-4646-b685-7e3d535d6b66" (UID: "352524ef-3b81-4646-b685-7e3d535d6b66"). InnerVolumeSpecName "kube-api-access-gkb5w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:54.324464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.324439    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b52gr\" (UniqueName: \"kubernetes.io/projected/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-kube-api-access-b52gr\") pod \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\" (UID: \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\") "
Sep 16 18:45:54.324627 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.324554    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-plugins-config-volume\") pod \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\" (UID: \"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8\") "
Sep 16 18:45:54.324864 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.324844    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkb5w\" (UniqueName: \"kubernetes.io/projected/352524ef-3b81-4646-b685-7e3d535d6b66-kube-api-access-gkb5w\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:54.324927 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.324869    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-maas-api-tls\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:54.324927 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.324884    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8vsk\" (UniqueName: \"kubernetes.io/projected/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-kube-api-access-q8vsk\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:54.324927 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.324900    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1ee0dd6-a874-4824-ba80-c2565efc8aa8-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:54.325091 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.324935    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a5a3d9c-6fce-4b7a-9593-ec822cee76b8" (UID: "1a5a3d9c-6fce-4b7a-9593-ec822cee76b8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:45:54.326489 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.326467    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-kube-api-access-b52gr" (OuterVolumeSpecName: "kube-api-access-b52gr") pod "1a5a3d9c-6fce-4b7a-9593-ec822cee76b8" (UID: "1a5a3d9c-6fce-4b7a-9593-ec822cee76b8"). InnerVolumeSpecName "kube-api-access-b52gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:45:54.425739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.425710    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:54.425739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.425736    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b52gr\" (UniqueName: \"kubernetes.io/projected/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8-kube-api-access-b52gr\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:45:54.899471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.899436    2576 generic.go:356] "Generic (PLEG): container finished" podID="1a5a3d9c-6fce-4b7a-9593-ec822cee76b8" containerID="96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8" exitCode=0
Sep 16 18:45:54.899904 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.899482    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"
Sep 16 18:45:54.899904 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.899519    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z" event={"ID":"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8","Type":"ContainerDied","Data":"96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8"}
Sep 16 18:45:54.899904 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.899580    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z" event={"ID":"1a5a3d9c-6fce-4b7a-9593-ec822cee76b8","Type":"ContainerDied","Data":"780485d23672af97641d807b9d9f911ba9c2d1db1df8894311978dbab1e9a9a5"}
Sep 16 18:45:54.899904 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.899617    2576 scope.go:117] "RemoveContainer" containerID="96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8"
Sep 16 18:45:54.901042 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.901021    2576 generic.go:356] "Generic (PLEG): container finished" podID="b1ee0dd6-a874-4824-ba80-c2565efc8aa8" containerID="853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0" exitCode=0
Sep 16 18:45:54.901112 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.901081    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"
Sep 16 18:45:54.901163 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.901118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn" event={"ID":"b1ee0dd6-a874-4824-ba80-c2565efc8aa8","Type":"ContainerDied","Data":"853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0"}
Sep 16 18:45:54.901244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.901170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn" event={"ID":"b1ee0dd6-a874-4824-ba80-c2565efc8aa8","Type":"ContainerDied","Data":"2b72f29f30fc4d533d051333131022bbc77e9e48501290baba0de2419a2055b2"}
Sep 16 18:45:54.902539 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.902518    2576 generic.go:356] "Generic (PLEG): container finished" podID="352524ef-3b81-4646-b685-7e3d535d6b66" containerID="cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606" exitCode=0
Sep 16 18:45:54.902641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.902553    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl" event={"ID":"352524ef-3b81-4646-b685-7e3d535d6b66","Type":"ContainerDied","Data":"cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606"}
Sep 16 18:45:54.902641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.902569    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"
Sep 16 18:45:54.902641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.902606    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl" event={"ID":"352524ef-3b81-4646-b685-7e3d535d6b66","Type":"ContainerDied","Data":"dbcf2564b3ca7bd17a9711e06e8256c48f00a39fa1ddc568eb0aec3d5b529695"}
Sep 16 18:45:54.914158 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.914030    2576 scope.go:117] "RemoveContainer" containerID="96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8"
Sep 16 18:45:54.914749 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:45:54.914721    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8\": container with ID starting with 96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8 not found: ID does not exist" containerID="96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8"
Sep 16 18:45:54.914844 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.914762    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8"} err="failed to get container status \"96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8\": rpc error: code = NotFound desc = could not find container \"96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8\": container with ID starting with 96fd50379421ce048a58ed42d81829531da82a295ccbfe61f1e37ab4cddb16b8 not found: ID does not exist"
Sep 16 18:45:54.914844 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.914790    2576 scope.go:117] "RemoveContainer" containerID="853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0"
Sep 16 18:45:54.923295 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.923273    2576 scope.go:117] "RemoveContainer" containerID="853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0"
Sep 16 18:45:54.923539 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:45:54.923518    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0\": container with ID starting with 853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0 not found: ID does not exist" containerID="853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0"
Sep 16 18:45:54.923591 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.923545    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0"} err="failed to get container status \"853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0\": rpc error: code = NotFound desc = could not find container \"853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0\": container with ID starting with 853f830aa8599e21d7dc4597a5cbdc72b29fbe21fbd8b53f69ac7acb8c0770c0 not found: ID does not exist"
Sep 16 18:45:54.923591 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.923561    2576 scope.go:117] "RemoveContainer" containerID="cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606"
Sep 16 18:45:54.931420 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.931398    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"]
Sep 16 18:45:54.933648 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.933633    2576 scope.go:117] "RemoveContainer" containerID="cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606"
Sep 16 18:45:54.934009 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:45:54.933982    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606\": container with ID starting with cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606 not found: ID does not exist" containerID="cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606"
Sep 16 18:45:54.934102 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.934016    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606"} err="failed to get container status \"cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606\": rpc error: code = NotFound desc = could not find container \"cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606\": container with ID starting with cf9373ff17e52386e62c6fedcf6ad70f823adc08f178d714e522db8677842606 not found: ID does not exist"
Sep 16 18:45:54.934603 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.934584    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z"]
Sep 16 18:45:54.945352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.945330    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a5a3d9c-6fce-4b7a-9593-ec822cee76b8" path="/var/lib/kubelet/pods/1a5a3d9c-6fce-4b7a-9593-ec822cee76b8/volumes"
Sep 16 18:45:54.945596 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.945584    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"]
Sep 16 18:45:54.956822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.953030    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-8edc30-565fc8c6b7-q7lvn"]
Sep 16 18:45:54.975641 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.975614    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"]
Sep 16 18:45:54.987818 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:54.987789    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl"]
Sep 16 18:45:56.945079 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:56.945049    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="352524ef-3b81-4646-b685-7e3d535d6b66" path="/var/lib/kubelet/pods/352524ef-3b81-4646-b685-7e3d535d6b66/volumes"
Sep 16 18:45:56.945563 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:45:56.945450    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1ee0dd6-a874-4824-ba80-c2565efc8aa8" path="/var/lib/kubelet/pods/b1ee0dd6-a874-4824-ba80-c2565efc8aa8/volumes"
Sep 16 18:46:55.712848 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:55.712814    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"]
Sep 16 18:46:55.713327 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:55.713016    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq" podUID="68a9f3b7-9a28-4795-b09f-2f19a402bd52" containerName="maas-api" containerID="cri-o://346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504" gracePeriod=30
Sep 16 18:46:55.827674 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:55.827643    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"]
Sep 16 18:46:55.827896 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:55.827869    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr" podUID="7e226d70-ee7b-4b2b-a348-68bb0d68c2ad" containerName="payload-pre-processing" containerID="cri-o://a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9" gracePeriod=30
Sep 16 18:46:55.975985 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:55.975930    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:46:56.001272 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.000794    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p5s2\" (UniqueName: \"kubernetes.io/projected/68a9f3b7-9a28-4795-b09f-2f19a402bd52-kube-api-access-7p5s2\") pod \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") "
Sep 16 18:46:56.001272 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.000875    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-maas-api-tls\") pod \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") "
Sep 16 18:46:56.001272 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.000945    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68a9f3b7-9a28-4795-b09f-2f19a402bd52-openshift-service-ca\") pod \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") "
Sep 16 18:46:56.001272 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.001014    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-metrics-certs\") pod \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\" (UID: \"68a9f3b7-9a28-4795-b09f-2f19a402bd52\") "
Sep 16 18:46:56.002234 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.002150    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68a9f3b7-9a28-4795-b09f-2f19a402bd52-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68a9f3b7-9a28-4795-b09f-2f19a402bd52" (UID: "68a9f3b7-9a28-4795-b09f-2f19a402bd52"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:46:56.005865 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.005792    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68a9f3b7-9a28-4795-b09f-2f19a402bd52-kube-api-access-7p5s2" (OuterVolumeSpecName: "kube-api-access-7p5s2") pod "68a9f3b7-9a28-4795-b09f-2f19a402bd52" (UID: "68a9f3b7-9a28-4795-b09f-2f19a402bd52"). InnerVolumeSpecName "kube-api-access-7p5s2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:46:56.007321 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.007294    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "68a9f3b7-9a28-4795-b09f-2f19a402bd52" (UID: "68a9f3b7-9a28-4795-b09f-2f19a402bd52"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:46:56.007462 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.007436    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68a9f3b7-9a28-4795-b09f-2f19a402bd52" (UID: "68a9f3b7-9a28-4795-b09f-2f19a402bd52"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:46:56.069907 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.069885    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:46:56.102565 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.102541    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tstxl\" (UniqueName: \"kubernetes.io/projected/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-kube-api-access-tstxl\") pod \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\" (UID: \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\") "
Sep 16 18:46:56.102694 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.102680    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-plugins-config-volume\") pod \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\" (UID: \"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad\") "
Sep 16 18:46:56.102930 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.102913    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68a9f3b7-9a28-4795-b09f-2f19a402bd52-openshift-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:46:56.103015 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.102932    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:46:56.103015 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.102942    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p5s2\" (UniqueName: \"kubernetes.io/projected/68a9f3b7-9a28-4795-b09f-2f19a402bd52-kube-api-access-7p5s2\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:46:56.103015 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.102951    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68a9f3b7-9a28-4795-b09f-2f19a402bd52-maas-api-tls\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:46:56.103154 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.103027    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7e226d70-ee7b-4b2b-a348-68bb0d68c2ad" (UID: "7e226d70-ee7b-4b2b-a348-68bb0d68c2ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:46:56.104585 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.104563    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-kube-api-access-tstxl" (OuterVolumeSpecName: "kube-api-access-tstxl") pod "7e226d70-ee7b-4b2b-a348-68bb0d68c2ad" (UID: "7e226d70-ee7b-4b2b-a348-68bb0d68c2ad"). InnerVolumeSpecName "kube-api-access-tstxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:46:56.203822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.203794    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:46:56.203822 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.203819    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tstxl\" (UniqueName: \"kubernetes.io/projected/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad-kube-api-access-tstxl\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:46:56.228105 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.228027    2576 generic.go:356] "Generic (PLEG): container finished" podID="68a9f3b7-9a28-4795-b09f-2f19a402bd52" containerID="346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504" exitCode=0
Sep 16 18:46:56.228105 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.228074    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"
Sep 16 18:46:56.228346 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.228118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq" event={"ID":"68a9f3b7-9a28-4795-b09f-2f19a402bd52","Type":"ContainerDied","Data":"346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504"}
Sep 16 18:46:56.228346 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.228169    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq" event={"ID":"68a9f3b7-9a28-4795-b09f-2f19a402bd52","Type":"ContainerDied","Data":"206c502722ddd74e40c3c6c0bb03b373604fe6091efa5c8331e93e5bc5832988"}
Sep 16 18:46:56.228346 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.228191    2576 scope.go:117] "RemoveContainer" containerID="346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504"
Sep 16 18:46:56.229513 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.229493    2576 generic.go:356] "Generic (PLEG): container finished" podID="7e226d70-ee7b-4b2b-a348-68bb0d68c2ad" containerID="a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9" exitCode=0
Sep 16 18:46:56.229583 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.229543    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr" event={"ID":"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad","Type":"ContainerDied","Data":"a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9"}
Sep 16 18:46:56.229630 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.229580    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr" event={"ID":"7e226d70-ee7b-4b2b-a348-68bb0d68c2ad","Type":"ContainerDied","Data":"c77d24784dee9448606eb93269cc08cb98c66a9e10b725e73de13effc00c6a84"}
Sep 16 18:46:56.229630 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.229548    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"
Sep 16 18:46:56.239101 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.239085    2576 scope.go:117] "RemoveContainer" containerID="346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504"
Sep 16 18:46:56.239441 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:46:56.239422    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504\": container with ID starting with 346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504 not found: ID does not exist" containerID="346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504"
Sep 16 18:46:56.239513 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.239450    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504"} err="failed to get container status \"346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504\": rpc error: code = NotFound desc = could not find container \"346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504\": container with ID starting with 346f758bfcd8baebc77c4ffe21ee3be18ce7f2e243a1c3f856f101b847329504 not found: ID does not exist"
Sep 16 18:46:56.239513 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.239471    2576 scope.go:117] "RemoveContainer" containerID="a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9"
Sep 16 18:46:56.247291 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.247253    2576 scope.go:117] "RemoveContainer" containerID="a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9"
Sep 16 18:46:56.247521 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:46:56.247504    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9\": container with ID starting with a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9 not found: ID does not exist" containerID="a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9"
Sep 16 18:46:56.247571 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.247528    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9"} err="failed to get container status \"a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9\": rpc error: code = NotFound desc = could not find container \"a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9\": container with ID starting with a0211dc03e8130688b98b6592da6e8dda6eef925ad43b6fd65874e1220675ab9 not found: ID does not exist"
Sep 16 18:46:56.286287 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.286262    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"]
Sep 16 18:46:56.293921 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.293902    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ee3af0-65f96c7bdc-q7dqq"]
Sep 16 18:46:56.317275 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.317245    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"]
Sep 16 18:46:56.323564 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.323538    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr"]
Sep 16 18:46:56.944638 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.944608    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68a9f3b7-9a28-4795-b09f-2f19a402bd52" path="/var/lib/kubelet/pods/68a9f3b7-9a28-4795-b09f-2f19a402bd52/volumes"
Sep 16 18:46:56.945003 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:46:56.944978    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e226d70-ee7b-4b2b-a348-68bb0d68c2ad" path="/var/lib/kubelet/pods/7e226d70-ee7b-4b2b-a348-68bb0d68c2ad/volumes"
Sep 16 18:47:17.259597 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:17.259567    2576 scope.go:117] "RemoveContainer" containerID="93b6ab267947034fbbbf19ecc3e6f4db4af6086ab589625d0dc4394203ea0577"
Sep 16 18:47:53.835241 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835155    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"]
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835623    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ee0dd6-a874-4824-ba80-c2565efc8aa8" containerName="maas-api"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835635    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ee0dd6-a874-4824-ba80-c2565efc8aa8" containerName="maas-api"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835649    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68a9f3b7-9a28-4795-b09f-2f19a402bd52" containerName="maas-api"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835655    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="68a9f3b7-9a28-4795-b09f-2f19a402bd52" containerName="maas-api"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835664    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="055da7ce-33bc-4ad9-a625-0d9eb6bd15b6" containerName="payload-pre-processing"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835670    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="055da7ce-33bc-4ad9-a625-0d9eb6bd15b6" containerName="payload-pre-processing"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835677    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b39d210-6738-4cf7-82e5-0f8025c69759" containerName="payload-processing"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835683    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b39d210-6738-4cf7-82e5-0f8025c69759" containerName="payload-processing"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835692    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="352524ef-3b81-4646-b685-7e3d535d6b66" containerName="payload-processing"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835697    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="352524ef-3b81-4646-b685-7e3d535d6b66" containerName="payload-processing"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835705    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a5a3d9c-6fce-4b7a-9593-ec822cee76b8" containerName="payload-pre-processing"
Sep 16 18:47:53.835712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835711    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a5a3d9c-6fce-4b7a-9593-ec822cee76b8" containerName="payload-pre-processing"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835721    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e48e7cb1-245c-4da0-9650-13188bcc50a0" containerName="main"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835728    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e48e7cb1-245c-4da0-9650-13188bcc50a0" containerName="main"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835742    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e226d70-ee7b-4b2b-a348-68bb0d68c2ad" containerName="payload-pre-processing"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835749    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e226d70-ee7b-4b2b-a348-68bb0d68c2ad" containerName="payload-pre-processing"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835819    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e226d70-ee7b-4b2b-a348-68bb0d68c2ad" containerName="payload-pre-processing"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835827    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a5a3d9c-6fce-4b7a-9593-ec822cee76b8" containerName="payload-pre-processing"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835834    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="352524ef-3b81-4646-b685-7e3d535d6b66" containerName="payload-processing"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835840    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b39d210-6738-4cf7-82e5-0f8025c69759" containerName="payload-processing"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835845    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="055da7ce-33bc-4ad9-a625-0d9eb6bd15b6" containerName="payload-pre-processing"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835851    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e48e7cb1-245c-4da0-9650-13188bcc50a0" containerName="main"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835858    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="68a9f3b7-9a28-4795-b09f-2f19a402bd52" containerName="maas-api"
Sep 16 18:47:53.836132 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.835864    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ee0dd6-a874-4824-ba80-c2565efc8aa8" containerName="maas-api"
Sep 16 18:47:53.838821 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.838803    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:53.842272 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.842250    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-1f16eb-serving-cert\""
Sep 16 18:47:53.862882 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.862858    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"]
Sep 16 18:47:53.955675 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.955642    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-metrics-certs\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:53.955675 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.955676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-maas-api-tls\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:53.955878 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.955713    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/428f273d-2237-4665-b934-037b999c9fa4-openshift-service-ca\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:53.955878 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:53.955830    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4s9x\" (UniqueName: \"kubernetes.io/projected/428f273d-2237-4665-b934-037b999c9fa4-kube-api-access-s4s9x\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.057083 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.057050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4s9x\" (UniqueName: \"kubernetes.io/projected/428f273d-2237-4665-b934-037b999c9fa4-kube-api-access-s4s9x\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.057377 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.057350    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-metrics-certs\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.057523 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.057449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-maas-api-tls\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.057595 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.057527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/428f273d-2237-4665-b934-037b999c9fa4-openshift-service-ca\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.057595 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:47:54.057543    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-serving-cert: secret "maas-api-e2e-api-a-1f16eb-serving-cert" not found
Sep 16 18:47:54.057687 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:47:54.057619    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-maas-api-tls podName:428f273d-2237-4665-b934-037b999c9fa4 nodeName:}" failed. No retries permitted until 2026-09-16 18:47:54.557599958 +0000 UTC m=+1898.113893252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-maas-api-tls") pod "maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6" (UID: "428f273d-2237-4665-b934-037b999c9fa4") : secret "maas-api-e2e-api-a-1f16eb-serving-cert" not found
Sep 16 18:47:54.058040 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.058011    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/428f273d-2237-4665-b934-037b999c9fa4-openshift-service-ca\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.059827 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.059810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-metrics-certs\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.070400 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.070379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4s9x\" (UniqueName: \"kubernetes.io/projected/428f273d-2237-4665-b934-037b999c9fa4-kube-api-access-s4s9x\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.562981 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.562942    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-maas-api-tls\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.565370 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.565349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-maas-api-tls\") pod \"maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:54.749759 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:54.749727    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:55.054470 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.054437    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"]
Sep 16 18:47:55.062455 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.062428    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:47:55.194881 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.194850    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"]
Sep 16 18:47:55.199907 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.199881    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:47:55.205864 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.205843    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-1f16eb-dockercfg-ww284\""
Sep 16 18:47:55.205998 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.205914    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-1f16eb\""
Sep 16 18:47:55.218140 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.218118    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"]
Sep 16 18:47:55.270450 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.270417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9zrx\" (UniqueName: \"kubernetes.io/projected/686e2780-ee89-49fe-a8b6-7fab0b539d76-kube-api-access-g9zrx\") pod \"payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp\" (UID: \"686e2780-ee89-49fe-a8b6-7fab0b539d76\") " pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:47:55.270600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.270491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/686e2780-ee89-49fe-a8b6-7fab0b539d76-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp\" (UID: \"686e2780-ee89-49fe-a8b6-7fab0b539d76\") " pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:47:55.371849 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.371780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9zrx\" (UniqueName: \"kubernetes.io/projected/686e2780-ee89-49fe-a8b6-7fab0b539d76-kube-api-access-g9zrx\") pod \"payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp\" (UID: \"686e2780-ee89-49fe-a8b6-7fab0b539d76\") " pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:47:55.371849 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.371832    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/686e2780-ee89-49fe-a8b6-7fab0b539d76-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp\" (UID: \"686e2780-ee89-49fe-a8b6-7fab0b539d76\") " pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:47:55.372512 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.372495    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/686e2780-ee89-49fe-a8b6-7fab0b539d76-plugins-config-volume\") pod \"payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp\" (UID: \"686e2780-ee89-49fe-a8b6-7fab0b539d76\") " pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:47:55.381746 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.381724    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9zrx\" (UniqueName: \"kubernetes.io/projected/686e2780-ee89-49fe-a8b6-7fab0b539d76-kube-api-access-g9zrx\") pod \"payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp\" (UID: \"686e2780-ee89-49fe-a8b6-7fab0b539d76\") " pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:47:55.511552 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.511517    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:47:55.531007 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.530973    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6" event={"ID":"428f273d-2237-4665-b934-037b999c9fa4","Type":"ContainerStarted","Data":"9796c4cc1ccced77abb0fd16afc311ef1da466132abaeccdab27750deb072fb4"}
Sep 16 18:47:55.785919 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:55.785897    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"]
Sep 16 18:47:55.787967 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:47:55.787932    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod686e2780_ee89_49fe_a8b6_7fab0b539d76.slice/crio-efeebf9e71216962d57a931aff49912e4decf93ecc10685c5dedab114758a70e WatchSource:0}: Error finding container efeebf9e71216962d57a931aff49912e4decf93ecc10685c5dedab114758a70e: Status 404 returned error can't find the container with id efeebf9e71216962d57a931aff49912e4decf93ecc10685c5dedab114758a70e
Sep 16 18:47:56.537669 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:56.537634    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6" event={"ID":"428f273d-2237-4665-b934-037b999c9fa4","Type":"ContainerStarted","Data":"2c97e61b50e160699f1da0258f02b669cb2e6be688fa2ffd162a4c609e740019"}
Sep 16 18:47:56.538135 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:56.537726    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:47:56.539365 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:56.539343    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp" event={"ID":"686e2780-ee89-49fe-a8b6-7fab0b539d76","Type":"ContainerStarted","Data":"b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9"}
Sep 16 18:47:56.539511 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:56.539385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp" event={"ID":"686e2780-ee89-49fe-a8b6-7fab0b539d76","Type":"ContainerStarted","Data":"efeebf9e71216962d57a931aff49912e4decf93ecc10685c5dedab114758a70e"}
Sep 16 18:47:56.539511 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:56.539414    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:47:56.563575 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:56.563526    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6" podStartSLOduration=2.99624152 podStartE2EDuration="3.563509757s" totalImagesPullingTime="567.268237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:56.560610538 +0000 UTC m=+1900.116903849" watchObservedRunningTime="2026-09-16 18:47:56.563509757 +0000 UTC m=+1900.119803068"
Sep 16 18:47:56.587988 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:47:56.587940    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp" podStartSLOduration=1.58792225 podStartE2EDuration="1.58792225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:47:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:56.585148112 +0000 UTC m=+1900.141441424" watchObservedRunningTime="2026-09-16 18:47:56.58792225 +0000 UTC m=+1900.144215565"
Sep 16 18:48:02.550104 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:02.550071    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:48:07.546427 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:07.546398    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:48:29.457297 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.457263    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"]
Sep 16 18:48:29.460854 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.460833    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:29.463401 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.463377    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-4dfacb\""
Sep 16 18:48:29.463544 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.463525    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-4dfacb-dockercfg-5w8qs\""
Sep 16 18:48:29.473048 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.473021    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"]
Sep 16 18:48:29.620922 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.620887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ds7s\" (UniqueName: \"kubernetes.io/projected/5cd8e01d-79f4-43a2-bf55-16dd582372f4-kube-api-access-5ds7s\") pod \"payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n\" (UID: \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:29.621098 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.620958    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cd8e01d-79f4-43a2-bf55-16dd582372f4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n\" (UID: \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:29.721692 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.721616    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cd8e01d-79f4-43a2-bf55-16dd582372f4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n\" (UID: \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:29.721839 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.721713    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ds7s\" (UniqueName: \"kubernetes.io/projected/5cd8e01d-79f4-43a2-bf55-16dd582372f4-kube-api-access-5ds7s\") pod \"payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n\" (UID: \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:29.722377 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.722358    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cd8e01d-79f4-43a2-bf55-16dd582372f4-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n\" (UID: \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:29.730072 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.730050    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ds7s\" (UniqueName: \"kubernetes.io/projected/5cd8e01d-79f4-43a2-bf55-16dd582372f4-kube-api-access-5ds7s\") pod \"payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n\" (UID: \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\") " pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:29.773106 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:29.773071    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:30.049664 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:30.049633    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"]
Sep 16 18:48:30.052025 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:48:30.051991    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd8e01d_79f4_43a2_bf55_16dd582372f4.slice/crio-3b0a6e5ca53ce0bf01456bf69b1828678801f1b8d5b84c7e38c43e182b85d111 WatchSource:0}: Error finding container 3b0a6e5ca53ce0bf01456bf69b1828678801f1b8d5b84c7e38c43e182b85d111: Status 404 returned error can't find the container with id 3b0a6e5ca53ce0bf01456bf69b1828678801f1b8d5b84c7e38c43e182b85d111
Sep 16 18:48:30.726690 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:30.726654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n" event={"ID":"5cd8e01d-79f4-43a2-bf55-16dd582372f4","Type":"ContainerStarted","Data":"246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470"}
Sep 16 18:48:30.727122 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:30.726704    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n" event={"ID":"5cd8e01d-79f4-43a2-bf55-16dd582372f4","Type":"ContainerStarted","Data":"3b0a6e5ca53ce0bf01456bf69b1828678801f1b8d5b84c7e38c43e182b85d111"}
Sep 16 18:48:30.727122 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:30.726808    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:30.745636 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:30.745583    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n" podStartSLOduration=1.7455650870000001 podStartE2EDuration="1.745565087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:48:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:48:30.743292966 +0000 UTC m=+1934.299586285" watchObservedRunningTime="2026-09-16 18:48:30.745565087 +0000 UTC m=+1934.301858398"
Sep 16 18:48:41.734979 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:41.734948    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:48:50.542916 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.542876    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"]
Sep 16 18:48:50.543770 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.543152    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf" podUID="613f3e0a-d507-45e5-8f3d-2237044c7ffd" containerName="payload-processing" containerID="cri-o://b59eeb367e60588a692e37a56e6071e1e7a333e426ef2e9f3f1c20d132923c56" gracePeriod=30
Sep 16 18:48:50.552058 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.552013    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"]
Sep 16 18:48:50.552977 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.552919    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln" podUID="6cb57348-cfbe-4f66-af79-7511d3eca1e2" containerName="payload-pre-processing" containerID="cri-o://77a8bb1b73c293fe6eb9ccc9ff7f064a8dedfee7780ccb52f263a672113c66a5" gracePeriod=30
Sep 16 18:48:50.838560 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.838530    2576 generic.go:356] "Generic (PLEG): container finished" podID="613f3e0a-d507-45e5-8f3d-2237044c7ffd" containerID="b59eeb367e60588a692e37a56e6071e1e7a333e426ef2e9f3f1c20d132923c56" exitCode=0
Sep 16 18:48:50.838711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.838584    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf" event={"ID":"613f3e0a-d507-45e5-8f3d-2237044c7ffd","Type":"ContainerDied","Data":"b59eeb367e60588a692e37a56e6071e1e7a333e426ef2e9f3f1c20d132923c56"}
Sep 16 18:48:50.838711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.838651    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf" event={"ID":"613f3e0a-d507-45e5-8f3d-2237044c7ffd","Type":"ContainerDied","Data":"82bc11d6fbc5615a1b3d21a1db05600e0cca801c47ed93dd1664a5c99d6ab5d1"}
Sep 16 18:48:50.838711 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.838677    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82bc11d6fbc5615a1b3d21a1db05600e0cca801c47ed93dd1664a5c99d6ab5d1"
Sep 16 18:48:50.839916 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.839896    2576 generic.go:356] "Generic (PLEG): container finished" podID="6cb57348-cfbe-4f66-af79-7511d3eca1e2" containerID="77a8bb1b73c293fe6eb9ccc9ff7f064a8dedfee7780ccb52f263a672113c66a5" exitCode=0
Sep 16 18:48:50.840021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.839940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln" event={"ID":"6cb57348-cfbe-4f66-af79-7511d3eca1e2","Type":"ContainerDied","Data":"77a8bb1b73c293fe6eb9ccc9ff7f064a8dedfee7780ccb52f263a672113c66a5"}
Sep 16 18:48:50.840021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.839971    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln" event={"ID":"6cb57348-cfbe-4f66-af79-7511d3eca1e2","Type":"ContainerDied","Data":"b63274bbeb1186649702401d99716f02570561582a90a509d580a34350c72714"}
Sep 16 18:48:50.840021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.839986    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b63274bbeb1186649702401d99716f02570561582a90a509d580a34350c72714"
Sep 16 18:48:50.841088 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.841068    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:48:50.844542 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.844525    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:48:50.929443 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.929416    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kx46\" (UniqueName: \"kubernetes.io/projected/613f3e0a-d507-45e5-8f3d-2237044c7ffd-kube-api-access-7kx46\") pod \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\" (UID: \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\") "
Sep 16 18:48:50.929612 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.929506    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/613f3e0a-d507-45e5-8f3d-2237044c7ffd-plugins-config-volume\") pod \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\" (UID: \"613f3e0a-d507-45e5-8f3d-2237044c7ffd\") "
Sep 16 18:48:50.929612 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.929526    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb57348-cfbe-4f66-af79-7511d3eca1e2-plugins-config-volume\") pod \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\" (UID: \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\") "
Sep 16 18:48:50.929612 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.929576    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx8t6\" (UniqueName: \"kubernetes.io/projected/6cb57348-cfbe-4f66-af79-7511d3eca1e2-kube-api-access-tx8t6\") pod \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\" (UID: \"6cb57348-cfbe-4f66-af79-7511d3eca1e2\") "
Sep 16 18:48:50.929908 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.929882    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/613f3e0a-d507-45e5-8f3d-2237044c7ffd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "613f3e0a-d507-45e5-8f3d-2237044c7ffd" (UID: "613f3e0a-d507-45e5-8f3d-2237044c7ffd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:50.930044 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.929946    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cb57348-cfbe-4f66-af79-7511d3eca1e2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6cb57348-cfbe-4f66-af79-7511d3eca1e2" (UID: "6cb57348-cfbe-4f66-af79-7511d3eca1e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:48:50.931611 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.931590    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/613f3e0a-d507-45e5-8f3d-2237044c7ffd-kube-api-access-7kx46" (OuterVolumeSpecName: "kube-api-access-7kx46") pod "613f3e0a-d507-45e5-8f3d-2237044c7ffd" (UID: "613f3e0a-d507-45e5-8f3d-2237044c7ffd"). InnerVolumeSpecName "kube-api-access-7kx46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:50.931723 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:50.931660    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cb57348-cfbe-4f66-af79-7511d3eca1e2-kube-api-access-tx8t6" (OuterVolumeSpecName: "kube-api-access-tx8t6") pod "6cb57348-cfbe-4f66-af79-7511d3eca1e2" (UID: "6cb57348-cfbe-4f66-af79-7511d3eca1e2"). InnerVolumeSpecName "kube-api-access-tx8t6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:48:51.032187 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.032150    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/613f3e0a-d507-45e5-8f3d-2237044c7ffd-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:48:51.032187 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.032182    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cb57348-cfbe-4f66-af79-7511d3eca1e2-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:48:51.032187 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.032192    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx8t6\" (UniqueName: \"kubernetes.io/projected/6cb57348-cfbe-4f66-af79-7511d3eca1e2-kube-api-access-tx8t6\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:48:51.032424 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.032200    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kx46\" (UniqueName: \"kubernetes.io/projected/613f3e0a-d507-45e5-8f3d-2237044c7ffd-kube-api-access-7kx46\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:48:51.844735 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.844701    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"
Sep 16 18:48:51.844735 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.844719    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"
Sep 16 18:48:51.867965 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.867934    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"]
Sep 16 18:48:51.875490 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.875460    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf"]
Sep 16 18:48:51.890905 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.890878    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"]
Sep 16 18:48:51.898154 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:51.898124    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln"]
Sep 16 18:48:52.945295 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:52.945254    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="613f3e0a-d507-45e5-8f3d-2237044c7ffd" path="/var/lib/kubelet/pods/613f3e0a-d507-45e5-8f3d-2237044c7ffd/volumes"
Sep 16 18:48:52.945845 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:48:52.945824    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cb57348-cfbe-4f66-af79-7511d3eca1e2" path="/var/lib/kubelet/pods/6cb57348-cfbe-4f66-af79-7511d3eca1e2/volumes"
Sep 16 18:49:03.751429 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:03.751396    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"]
Sep 16 18:49:03.751876 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:03.751614    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6" podUID="428f273d-2237-4665-b934-037b999c9fa4" containerName="maas-api" containerID="cri-o://2c97e61b50e160699f1da0258f02b669cb2e6be688fa2ffd162a4c609e740019" gracePeriod=30
Sep 16 18:49:03.856734 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:03.856702    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"]
Sep 16 18:49:03.856955 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:03.856928    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp" podUID="686e2780-ee89-49fe-a8b6-7fab0b539d76" containerName="payload-processing" containerID="cri-o://b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9" gracePeriod=30
Sep 16 18:49:03.910904 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:03.910872    2576 generic.go:356] "Generic (PLEG): container finished" podID="428f273d-2237-4665-b934-037b999c9fa4" containerID="2c97e61b50e160699f1da0258f02b669cb2e6be688fa2ffd162a4c609e740019" exitCode=0
Sep 16 18:49:03.911058 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:03.910940    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6" event={"ID":"428f273d-2237-4665-b934-037b999c9fa4","Type":"ContainerDied","Data":"2c97e61b50e160699f1da0258f02b669cb2e6be688fa2ffd162a4c609e740019"}
Sep 16 18:49:04.010428 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.010366    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:49:04.057218 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.057168    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-maas-api-tls\") pod \"428f273d-2237-4665-b934-037b999c9fa4\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") "
Sep 16 18:49:04.057363 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.057249    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/428f273d-2237-4665-b934-037b999c9fa4-openshift-service-ca\") pod \"428f273d-2237-4665-b934-037b999c9fa4\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") "
Sep 16 18:49:04.057363 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.057315    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4s9x\" (UniqueName: \"kubernetes.io/projected/428f273d-2237-4665-b934-037b999c9fa4-kube-api-access-s4s9x\") pod \"428f273d-2237-4665-b934-037b999c9fa4\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") "
Sep 16 18:49:04.057363 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.057349    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-metrics-certs\") pod \"428f273d-2237-4665-b934-037b999c9fa4\" (UID: \"428f273d-2237-4665-b934-037b999c9fa4\") "
Sep 16 18:49:04.057911 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.057874    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/428f273d-2237-4665-b934-037b999c9fa4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "428f273d-2237-4665-b934-037b999c9fa4" (UID: "428f273d-2237-4665-b934-037b999c9fa4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:04.058120 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.058096    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/428f273d-2237-4665-b934-037b999c9fa4-openshift-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:04.059769 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.059739    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/428f273d-2237-4665-b934-037b999c9fa4-kube-api-access-s4s9x" (OuterVolumeSpecName: "kube-api-access-s4s9x") pod "428f273d-2237-4665-b934-037b999c9fa4" (UID: "428f273d-2237-4665-b934-037b999c9fa4"). InnerVolumeSpecName "kube-api-access-s4s9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:04.060135 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.060111    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "428f273d-2237-4665-b934-037b999c9fa4" (UID: "428f273d-2237-4665-b934-037b999c9fa4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:04.060244 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.060143    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "428f273d-2237-4665-b934-037b999c9fa4" (UID: "428f273d-2237-4665-b934-037b999c9fa4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:04.109428 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.109394    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:49:04.158978 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.158944    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/686e2780-ee89-49fe-a8b6-7fab0b539d76-plugins-config-volume\") pod \"686e2780-ee89-49fe-a8b6-7fab0b539d76\" (UID: \"686e2780-ee89-49fe-a8b6-7fab0b539d76\") "
Sep 16 18:49:04.159164 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.159065    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9zrx\" (UniqueName: \"kubernetes.io/projected/686e2780-ee89-49fe-a8b6-7fab0b539d76-kube-api-access-g9zrx\") pod \"686e2780-ee89-49fe-a8b6-7fab0b539d76\" (UID: \"686e2780-ee89-49fe-a8b6-7fab0b539d76\") "
Sep 16 18:49:04.159393 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.159370    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/686e2780-ee89-49fe-a8b6-7fab0b539d76-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "686e2780-ee89-49fe-a8b6-7fab0b539d76" (UID: "686e2780-ee89-49fe-a8b6-7fab0b539d76"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:04.159468 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.159451    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-maas-api-tls\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:04.159525 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.159477    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4s9x\" (UniqueName: \"kubernetes.io/projected/428f273d-2237-4665-b934-037b999c9fa4-kube-api-access-s4s9x\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:04.159525 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.159492    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/428f273d-2237-4665-b934-037b999c9fa4-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:04.161338 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.161315    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/686e2780-ee89-49fe-a8b6-7fab0b539d76-kube-api-access-g9zrx" (OuterVolumeSpecName: "kube-api-access-g9zrx") pod "686e2780-ee89-49fe-a8b6-7fab0b539d76" (UID: "686e2780-ee89-49fe-a8b6-7fab0b539d76"). InnerVolumeSpecName "kube-api-access-g9zrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:04.260823 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.260721    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/686e2780-ee89-49fe-a8b6-7fab0b539d76-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:04.260823 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.260759    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9zrx\" (UniqueName: \"kubernetes.io/projected/686e2780-ee89-49fe-a8b6-7fab0b539d76-kube-api-access-g9zrx\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:04.918627 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.918589    2576 generic.go:356] "Generic (PLEG): container finished" podID="686e2780-ee89-49fe-a8b6-7fab0b539d76" containerID="b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9" exitCode=0
Sep 16 18:49:04.919093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.918640    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"
Sep 16 18:49:04.919093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.918676    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp" event={"ID":"686e2780-ee89-49fe-a8b6-7fab0b539d76","Type":"ContainerDied","Data":"b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9"}
Sep 16 18:49:04.919093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.918737    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp" event={"ID":"686e2780-ee89-49fe-a8b6-7fab0b539d76","Type":"ContainerDied","Data":"efeebf9e71216962d57a931aff49912e4decf93ecc10685c5dedab114758a70e"}
Sep 16 18:49:04.919093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.918768    2576 scope.go:117] "RemoveContainer" containerID="b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9"
Sep 16 18:49:04.921464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.921442    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6" event={"ID":"428f273d-2237-4665-b934-037b999c9fa4","Type":"ContainerDied","Data":"9796c4cc1ccced77abb0fd16afc311ef1da466132abaeccdab27750deb072fb4"}
Sep 16 18:49:04.921607 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.921480    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"
Sep 16 18:49:04.929301 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.929280    2576 scope.go:117] "RemoveContainer" containerID="b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9"
Sep 16 18:49:04.929647 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:49:04.929618    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9\": container with ID starting with b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9 not found: ID does not exist" containerID="b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9"
Sep 16 18:49:04.929714 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.929660    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9"} err="failed to get container status \"b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9\": rpc error: code = NotFound desc = could not find container \"b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9\": container with ID starting with b829514bef1234711d48d81cb7045b7817753e5d3ae543437385897359ba8fc9 not found: ID does not exist"
Sep 16 18:49:04.929714 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.929684    2576 scope.go:117] "RemoveContainer" containerID="2c97e61b50e160699f1da0258f02b669cb2e6be688fa2ffd162a4c609e740019"
Sep 16 18:49:04.950904 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.950881    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"]
Sep 16 18:49:04.958307 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.958285    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp"]
Sep 16 18:49:04.970739 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.970716    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"]
Sep 16 18:49:04.976097 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:04.976076    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-1f16eb-768fbf5d79-2g6v6"]
Sep 16 18:49:06.944418 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:06.944385    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="428f273d-2237-4665-b934-037b999c9fa4" path="/var/lib/kubelet/pods/428f273d-2237-4665-b934-037b999c9fa4/volumes"
Sep 16 18:49:06.944867 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:06.944762    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="686e2780-ee89-49fe-a8b6-7fab0b539d76" path="/var/lib/kubelet/pods/686e2780-ee89-49fe-a8b6-7fab0b539d76/volumes"
Sep 16 18:49:17.295787 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:17.295710    2576 scope.go:117] "RemoveContainer" containerID="b59eeb367e60588a692e37a56e6071e1e7a333e426ef2e9f3f1c20d132923c56"
Sep 16 18:49:17.305059 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:17.305040    2576 scope.go:117] "RemoveContainer" containerID="77a8bb1b73c293fe6eb9ccc9ff7f064a8dedfee7780ccb52f263a672113c66a5"
Sep 16 18:49:27.360352 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:27.360313    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"]
Sep 16 18:49:27.360954 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:27.360586    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n" podUID="5cd8e01d-79f4-43a2-bf55-16dd582372f4" containerName="payload-processing" containerID="cri-o://246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470" gracePeriod=30
Sep 16 18:49:27.614389 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:27.614320    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:49:27.685124 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:27.685090    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ds7s\" (UniqueName: \"kubernetes.io/projected/5cd8e01d-79f4-43a2-bf55-16dd582372f4-kube-api-access-5ds7s\") pod \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\" (UID: \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\") "
Sep 16 18:49:27.685330 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:27.685179    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cd8e01d-79f4-43a2-bf55-16dd582372f4-plugins-config-volume\") pod \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\" (UID: \"5cd8e01d-79f4-43a2-bf55-16dd582372f4\") "
Sep 16 18:49:27.685588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:27.685559    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cd8e01d-79f4-43a2-bf55-16dd582372f4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5cd8e01d-79f4-43a2-bf55-16dd582372f4" (UID: "5cd8e01d-79f4-43a2-bf55-16dd582372f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:27.687277 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:27.687251    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cd8e01d-79f4-43a2-bf55-16dd582372f4-kube-api-access-5ds7s" (OuterVolumeSpecName: "kube-api-access-5ds7s") pod "5cd8e01d-79f4-43a2-bf55-16dd582372f4" (UID: "5cd8e01d-79f4-43a2-bf55-16dd582372f4"). InnerVolumeSpecName "kube-api-access-5ds7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:27.786764 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:27.786732    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ds7s\" (UniqueName: \"kubernetes.io/projected/5cd8e01d-79f4-43a2-bf55-16dd582372f4-kube-api-access-5ds7s\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:27.786764 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:27.786760    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5cd8e01d-79f4-43a2-bf55-16dd582372f4-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:28.045512 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.045475    2576 generic.go:356] "Generic (PLEG): container finished" podID="5cd8e01d-79f4-43a2-bf55-16dd582372f4" containerID="246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470" exitCode=0
Sep 16 18:49:28.045673 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.045523    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"
Sep 16 18:49:28.045673 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.045542    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n" event={"ID":"5cd8e01d-79f4-43a2-bf55-16dd582372f4","Type":"ContainerDied","Data":"246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470"}
Sep 16 18:49:28.045673 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.045595    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n" event={"ID":"5cd8e01d-79f4-43a2-bf55-16dd582372f4","Type":"ContainerDied","Data":"3b0a6e5ca53ce0bf01456bf69b1828678801f1b8d5b84c7e38c43e182b85d111"}
Sep 16 18:49:28.045673 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.045625    2576 scope.go:117] "RemoveContainer" containerID="246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470"
Sep 16 18:49:28.054574 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.054557    2576 scope.go:117] "RemoveContainer" containerID="246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470"
Sep 16 18:49:28.054827 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:49:28.054810    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470\": container with ID starting with 246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470 not found: ID does not exist" containerID="246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470"
Sep 16 18:49:28.054868 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.054836    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470"} err="failed to get container status \"246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470\": rpc error: code = NotFound desc = could not find container \"246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470\": container with ID starting with 246e5c986a865886dd42974d217a2362072c09ee12e3ebc35728fab2ed998470 not found: ID does not exist"
Sep 16 18:49:28.077035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.076996    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"]
Sep 16 18:49:28.082013 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.081985    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n"]
Sep 16 18:49:28.946080 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:28.946047    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cd8e01d-79f4-43a2-bf55-16dd582372f4" path="/var/lib/kubelet/pods/5cd8e01d-79f4-43a2-bf55-16dd582372f4/volumes"
Sep 16 18:49:45.274411 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.274380    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"]
Sep 16 18:49:45.274888 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.274637    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc" podUID="553108f0-a26a-4a43-a0a3-406c8493b4ca" containerName="maas-api" containerID="cri-o://08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003" gracePeriod=30
Sep 16 18:49:45.369372 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.369338    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"]
Sep 16 18:49:45.369573 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.369537    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9" podUID="d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a" containerName="payload-pre-processing" containerID="cri-o://75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d" gracePeriod=30
Sep 16 18:49:45.545122 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.545093    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:49:45.641124 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.641101    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:49:45.656818 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.656788    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/553108f0-a26a-4a43-a0a3-406c8493b4ca-openshift-service-ca\") pod \"553108f0-a26a-4a43-a0a3-406c8493b4ca\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") "
Sep 16 18:49:45.656959 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.656850    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-maas-api-tls\") pod \"553108f0-a26a-4a43-a0a3-406c8493b4ca\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") "
Sep 16 18:49:45.657035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.656952    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkqlr\" (UniqueName: \"kubernetes.io/projected/553108f0-a26a-4a43-a0a3-406c8493b4ca-kube-api-access-tkqlr\") pod \"553108f0-a26a-4a43-a0a3-406c8493b4ca\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") "
Sep 16 18:49:45.657035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.657005    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-metrics-certs\") pod \"553108f0-a26a-4a43-a0a3-406c8493b4ca\" (UID: \"553108f0-a26a-4a43-a0a3-406c8493b4ca\") "
Sep 16 18:49:45.657198 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.657177    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/553108f0-a26a-4a43-a0a3-406c8493b4ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "553108f0-a26a-4a43-a0a3-406c8493b4ca" (UID: "553108f0-a26a-4a43-a0a3-406c8493b4ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:45.657505 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.657481    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/553108f0-a26a-4a43-a0a3-406c8493b4ca-openshift-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:45.659110 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.659073    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/553108f0-a26a-4a43-a0a3-406c8493b4ca-kube-api-access-tkqlr" (OuterVolumeSpecName: "kube-api-access-tkqlr") pod "553108f0-a26a-4a43-a0a3-406c8493b4ca" (UID: "553108f0-a26a-4a43-a0a3-406c8493b4ca"). InnerVolumeSpecName "kube-api-access-tkqlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:45.659225 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.659158    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "553108f0-a26a-4a43-a0a3-406c8493b4ca" (UID: "553108f0-a26a-4a43-a0a3-406c8493b4ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:45.659464 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.659442    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "553108f0-a26a-4a43-a0a3-406c8493b4ca" (UID: "553108f0-a26a-4a43-a0a3-406c8493b4ca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:49:45.757851 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.757821    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-plugins-config-volume\") pod \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\" (UID: \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\") "
Sep 16 18:49:45.758029 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.757950    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrmxg\" (UniqueName: \"kubernetes.io/projected/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-kube-api-access-hrmxg\") pod \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\" (UID: \"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a\") "
Sep 16 18:49:45.758290 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.758263    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a" (UID: "d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:49:45.758360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.758273    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-maas-api-tls\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:45.758360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.758325    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkqlr\" (UniqueName: \"kubernetes.io/projected/553108f0-a26a-4a43-a0a3-406c8493b4ca-kube-api-access-tkqlr\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:45.758360 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.758338    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/553108f0-a26a-4a43-a0a3-406c8493b4ca-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:45.760083 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.760064    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-kube-api-access-hrmxg" (OuterVolumeSpecName: "kube-api-access-hrmxg") pod "d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a" (UID: "d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a"). InnerVolumeSpecName "kube-api-access-hrmxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:49:45.859187 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.859117    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrmxg\" (UniqueName: \"kubernetes.io/projected/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-kube-api-access-hrmxg\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:45.859187 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:45.859142    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:49:46.138799 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.138711    2576 generic.go:356] "Generic (PLEG): container finished" podID="d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a" containerID="75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d" exitCode=0
Sep 16 18:49:46.138799 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.138753    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"
Sep 16 18:49:46.139024 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.138800    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9" event={"ID":"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a","Type":"ContainerDied","Data":"75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d"}
Sep 16 18:49:46.139024 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.138857    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9" event={"ID":"d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a","Type":"ContainerDied","Data":"ccb8052e8d5aebcc2301f6ea44ed6715a7a6d155ceb429916e97ef66bc23c634"}
Sep 16 18:49:46.139024 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.138888    2576 scope.go:117] "RemoveContainer" containerID="75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d"
Sep 16 18:49:46.140234 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.140187    2576 generic.go:356] "Generic (PLEG): container finished" podID="553108f0-a26a-4a43-a0a3-406c8493b4ca" containerID="08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003" exitCode=0
Sep 16 18:49:46.140343 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.140242    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"
Sep 16 18:49:46.140343 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.140247    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc" event={"ID":"553108f0-a26a-4a43-a0a3-406c8493b4ca","Type":"ContainerDied","Data":"08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003"}
Sep 16 18:49:46.140343 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.140322    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc" event={"ID":"553108f0-a26a-4a43-a0a3-406c8493b4ca","Type":"ContainerDied","Data":"6ab5fe11c87f71fb5359b8c8a8b75624919b6d006a5101a3e3699eb725b4010f"}
Sep 16 18:49:46.149312 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.149251    2576 scope.go:117] "RemoveContainer" containerID="75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d"
Sep 16 18:49:46.149580 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:49:46.149559    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d\": container with ID starting with 75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d not found: ID does not exist" containerID="75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d"
Sep 16 18:49:46.149686 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.149587    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d"} err="failed to get container status \"75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d\": rpc error: code = NotFound desc = could not find container \"75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d\": container with ID starting with 75bd50fc80d03c32f93f56f25d6d9cd8987fb2af7fd3b23fce68be4cde1ad59d not found: ID does not exist"
Sep 16 18:49:46.149686 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.149603    2576 scope.go:117] "RemoveContainer" containerID="08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003"
Sep 16 18:49:46.157934 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.157918    2576 scope.go:117] "RemoveContainer" containerID="08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003"
Sep 16 18:49:46.158217 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:49:46.158183    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003\": container with ID starting with 08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003 not found: ID does not exist" containerID="08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003"
Sep 16 18:49:46.158279 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.158227    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003"} err="failed to get container status \"08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003\": rpc error: code = NotFound desc = could not find container \"08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003\": container with ID starting with 08e8e9cc79ab0d5841a7548989b8b2f54775b6be0f54c8f47fb815e91c34c003 not found: ID does not exist"
Sep 16 18:49:46.172035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.171992    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"]
Sep 16 18:49:46.174631 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.174601    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-04899e-5c869b5dd8-gl9jc"]
Sep 16 18:49:46.191428 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.191409    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"]
Sep 16 18:49:46.197774 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.197756    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9"]
Sep 16 18:49:46.945276 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.945246    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="553108f0-a26a-4a43-a0a3-406c8493b4ca" path="/var/lib/kubelet/pods/553108f0-a26a-4a43-a0a3-406c8493b4ca/volumes"
Sep 16 18:49:46.945663 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:49:46.945643    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a" path="/var/lib/kubelet/pods/d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a/volumes"
Sep 16 18:50:03.775478 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:03.775439    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"]
Sep 16 18:50:03.776035 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:03.775737    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8" podUID="d7675640-fd5f-49d1-9711-c4deb19aaa88" containerName="maas-api" containerID="cri-o://26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f" gracePeriod=30
Sep 16 18:50:03.850199 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:03.850166    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"]
Sep 16 18:50:03.850416 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:03.850375    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29" podUID="9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4" containerName="payload-pre-processing" containerID="cri-o://97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904" gracePeriod=30
Sep 16 18:50:04.122748 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.122720    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:50:04.126455 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.126431    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:50:04.231842 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.231809    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-metrics-certs\") pod \"d7675640-fd5f-49d1-9711-c4deb19aaa88\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") "
Sep 16 18:50:04.231842 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.231847    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6q9q\" (UniqueName: \"kubernetes.io/projected/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-kube-api-access-c6q9q\") pod \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\" (UID: \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\") "
Sep 16 18:50:04.232069 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.231875    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-plugins-config-volume\") pod \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\" (UID: \"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4\") "
Sep 16 18:50:04.232069 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.231966    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2s9m\" (UniqueName: \"kubernetes.io/projected/d7675640-fd5f-49d1-9711-c4deb19aaa88-kube-api-access-g2s9m\") pod \"d7675640-fd5f-49d1-9711-c4deb19aaa88\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") "
Sep 16 18:50:04.232069 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.231997    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-maas-api-tls\") pod \"d7675640-fd5f-49d1-9711-c4deb19aaa88\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") "
Sep 16 18:50:04.232069 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.232030    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7675640-fd5f-49d1-9711-c4deb19aaa88-openshift-service-ca\") pod \"d7675640-fd5f-49d1-9711-c4deb19aaa88\" (UID: \"d7675640-fd5f-49d1-9711-c4deb19aaa88\") "
Sep 16 18:50:04.232424 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.232393    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4" (UID: "9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:50:04.232540 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.232488    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7675640-fd5f-49d1-9711-c4deb19aaa88-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7675640-fd5f-49d1-9711-c4deb19aaa88" (UID: "d7675640-fd5f-49d1-9711-c4deb19aaa88"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:50:04.234180 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.234157    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d7675640-fd5f-49d1-9711-c4deb19aaa88" (UID: "d7675640-fd5f-49d1-9711-c4deb19aaa88"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:50:04.234180 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.234164    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-kube-api-access-c6q9q" (OuterVolumeSpecName: "kube-api-access-c6q9q") pod "9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4" (UID: "9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4"). InnerVolumeSpecName "kube-api-access-c6q9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:50:04.234509 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.234489    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7675640-fd5f-49d1-9711-c4deb19aaa88-kube-api-access-g2s9m" (OuterVolumeSpecName: "kube-api-access-g2s9m") pod "d7675640-fd5f-49d1-9711-c4deb19aaa88" (UID: "d7675640-fd5f-49d1-9711-c4deb19aaa88"). InnerVolumeSpecName "kube-api-access-g2s9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:50:04.234580 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.234490    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7675640-fd5f-49d1-9711-c4deb19aaa88" (UID: "d7675640-fd5f-49d1-9711-c4deb19aaa88"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:50:04.242258 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.242230    2576 generic.go:356] "Generic (PLEG): container finished" podID="9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4" containerID="97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904" exitCode=0
Sep 16 18:50:04.242368 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.242282    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"
Sep 16 18:50:04.242368 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.242317    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29" event={"ID":"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4","Type":"ContainerDied","Data":"97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904"}
Sep 16 18:50:04.242485 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.242378    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29" event={"ID":"9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4","Type":"ContainerDied","Data":"274823cb14c0fac8d10c4b1de1b541fd177eedd238f50cf8ca09a6e503783df0"}
Sep 16 18:50:04.242485 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.242407    2576 scope.go:117] "RemoveContainer" containerID="97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904"
Sep 16 18:50:04.244013 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.243988    2576 generic.go:356] "Generic (PLEG): container finished" podID="d7675640-fd5f-49d1-9711-c4deb19aaa88" containerID="26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f" exitCode=0
Sep 16 18:50:04.244100 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.244034    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"
Sep 16 18:50:04.244100 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.244067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8" event={"ID":"d7675640-fd5f-49d1-9711-c4deb19aaa88","Type":"ContainerDied","Data":"26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f"}
Sep 16 18:50:04.244223 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.244114    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8" event={"ID":"d7675640-fd5f-49d1-9711-c4deb19aaa88","Type":"ContainerDied","Data":"b3805d9cf400c428f3ed6491fc38898d9b5915791d57f4cf3133175303958bde"}
Sep 16 18:50:04.253790 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.253768    2576 scope.go:117] "RemoveContainer" containerID="97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904"
Sep 16 18:50:04.254116 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:50:04.254096    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904\": container with ID starting with 97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904 not found: ID does not exist" containerID="97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904"
Sep 16 18:50:04.254249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.254126    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904"} err="failed to get container status \"97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904\": rpc error: code = NotFound desc = could not find container \"97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904\": container with ID starting with 97aa71e30029e3460ac8ffa15b9a157c7b5d655f4b4410c3368339eb9e49a904 not found: ID does not exist"
Sep 16 18:50:04.254249 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.254150    2576 scope.go:117] "RemoveContainer" containerID="26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f"
Sep 16 18:50:04.262857 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.262839    2576 scope.go:117] "RemoveContainer" containerID="26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f"
Sep 16 18:50:04.263092 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:50:04.263075    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f\": container with ID starting with 26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f not found: ID does not exist" containerID="26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f"
Sep 16 18:50:04.263192 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.263097    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f"} err="failed to get container status \"26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f\": rpc error: code = NotFound desc = could not find container \"26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f\": container with ID starting with 26ef7f2d2ecdc346e5db6457ad39c836453f459d1bcb1bf335b67a9947125c6f not found: ID does not exist"
Sep 16 18:50:04.273191 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.273166    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"]
Sep 16 18:50:04.276461 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.276440    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-520999-8689cf9cf4-fb8g8"]
Sep 16 18:50:04.292337 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.292314    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"]
Sep 16 18:50:04.295502 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.295479    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29"]
Sep 16 18:50:04.333789 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.333715    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2s9m\" (UniqueName: \"kubernetes.io/projected/d7675640-fd5f-49d1-9711-c4deb19aaa88-kube-api-access-g2s9m\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:50:04.333789 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.333742    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-maas-api-tls\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:50:04.333789 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.333753    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7675640-fd5f-49d1-9711-c4deb19aaa88-openshift-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:50:04.333789 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.333762    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7675640-fd5f-49d1-9711-c4deb19aaa88-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:50:04.333789 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.333771    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6q9q\" (UniqueName: \"kubernetes.io/projected/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-kube-api-access-c6q9q\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:50:04.333789 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.333779    2576 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4-plugins-config-volume\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:50:04.944732 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.944697    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4" path="/var/lib/kubelet/pods/9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4/volumes"
Sep 16 18:50:04.945135 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:04.945122    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7675640-fd5f-49d1-9711-c4deb19aaa88" path="/var/lib/kubelet/pods/d7675640-fd5f-49d1-9711-c4deb19aaa88/volumes"
Sep 16 18:50:15.833743 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:15.833703    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c46454c74-4t6h9"]
Sep 16 18:50:15.834182 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:15.833926    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c46454c74-4t6h9" podUID="159f0984-3a84-424b-a94b-f477ea325155" containerName="manager" containerID="cri-o://e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739" gracePeriod=30
Sep 16 18:50:16.073239 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.073198    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:50:16.144532 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.144506    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-metrics-certs\") pod \"159f0984-3a84-424b-a94b-f477ea325155\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") "
Sep 16 18:50:16.144708 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.144587    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsjdm\" (UniqueName: \"kubernetes.io/projected/159f0984-3a84-424b-a94b-f477ea325155-kube-api-access-dsjdm\") pod \"159f0984-3a84-424b-a94b-f477ea325155\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") "
Sep 16 18:50:16.144708 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.144657    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-webhook-cert\") pod \"159f0984-3a84-424b-a94b-f477ea325155\" (UID: \"159f0984-3a84-424b-a94b-f477ea325155\") "
Sep 16 18:50:16.146678 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.146645    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "159f0984-3a84-424b-a94b-f477ea325155" (UID: "159f0984-3a84-424b-a94b-f477ea325155"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:50:16.146809 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.146683    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/159f0984-3a84-424b-a94b-f477ea325155-kube-api-access-dsjdm" (OuterVolumeSpecName: "kube-api-access-dsjdm") pod "159f0984-3a84-424b-a94b-f477ea325155" (UID: "159f0984-3a84-424b-a94b-f477ea325155"). InnerVolumeSpecName "kube-api-access-dsjdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:50:16.146809 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.146736    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "159f0984-3a84-424b-a94b-f477ea325155" (UID: "159f0984-3a84-424b-a94b-f477ea325155"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:50:16.245680 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.245637    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:50:16.245680 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.245677    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsjdm\" (UniqueName: \"kubernetes.io/projected/159f0984-3a84-424b-a94b-f477ea325155-kube-api-access-dsjdm\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:50:16.245680 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.245687    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/159f0984-3a84-424b-a94b-f477ea325155-webhook-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:50:16.308071 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.308036    2576 generic.go:356] "Generic (PLEG): container finished" podID="159f0984-3a84-424b-a94b-f477ea325155" containerID="e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739" exitCode=0
Sep 16 18:50:16.308238 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.308071    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-4t6h9" event={"ID":"159f0984-3a84-424b-a94b-f477ea325155","Type":"ContainerDied","Data":"e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739"}
Sep 16 18:50:16.308238 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.308082    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-4t6h9"
Sep 16 18:50:16.308238 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.308138    2576 scope.go:117] "RemoveContainer" containerID="e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739"
Sep 16 18:50:16.308238 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.308127    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-4t6h9" event={"ID":"159f0984-3a84-424b-a94b-f477ea325155","Type":"ContainerDied","Data":"65b791b286b8890a8dba42ad67b42534bdaa67ad5ed8888c2d20760c4d5981c0"}
Sep 16 18:50:16.318748 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.318731    2576 scope.go:117] "RemoveContainer" containerID="e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739"
Sep 16 18:50:16.319017 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:50:16.318995    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739\": container with ID starting with e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739 not found: ID does not exist" containerID="e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739"
Sep 16 18:50:16.319069 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.319027    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739"} err="failed to get container status \"e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739\": rpc error: code = NotFound desc = could not find container \"e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739\": container with ID starting with e51608dfcafaa7bb342efa3a27280832fab3bba15a361887a795cec60d5f0739 not found: ID does not exist"
Sep 16 18:50:16.332042 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.332021    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c46454c74-4t6h9"]
Sep 16 18:50:16.333748 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.333718    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c46454c74-4t6h9"]
Sep 16 18:50:16.944563 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:50:16.944529    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="159f0984-3a84-424b-a94b-f477ea325155" path="/var/lib/kubelet/pods/159f0984-3a84-424b-a94b-f477ea325155/volumes"
Sep 16 18:51:17.156079 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156048    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c46454c74-x5p86"]
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156466    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cb57348-cfbe-4f66-af79-7511d3eca1e2" containerName="payload-pre-processing"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156478    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cb57348-cfbe-4f66-af79-7511d3eca1e2" containerName="payload-pre-processing"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156492    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a" containerName="payload-pre-processing"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156501    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a" containerName="payload-pre-processing"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156510    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159f0984-3a84-424b-a94b-f477ea325155" containerName="manager"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156515    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="159f0984-3a84-424b-a94b-f477ea325155" containerName="manager"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156524    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7675640-fd5f-49d1-9711-c4deb19aaa88" containerName="maas-api"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156530    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7675640-fd5f-49d1-9711-c4deb19aaa88" containerName="maas-api"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156536    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="553108f0-a26a-4a43-a0a3-406c8493b4ca" containerName="maas-api"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156542    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="553108f0-a26a-4a43-a0a3-406c8493b4ca" containerName="maas-api"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156549    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4" containerName="payload-pre-processing"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156554    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4" containerName="payload-pre-processing"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156566    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613f3e0a-d507-45e5-8f3d-2237044c7ffd" containerName="payload-processing"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156571    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="613f3e0a-d507-45e5-8f3d-2237044c7ffd" containerName="payload-processing"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156578    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="428f273d-2237-4665-b934-037b999c9fa4" containerName="maas-api"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156583    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="428f273d-2237-4665-b934-037b999c9fa4" containerName="maas-api"
Sep 16 18:51:17.156588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156598    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="686e2780-ee89-49fe-a8b6-7fab0b539d76" containerName="payload-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156603    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="686e2780-ee89-49fe-a8b6-7fab0b539d76" containerName="payload-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156611    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cd8e01d-79f4-43a2-bf55-16dd582372f4" containerName="payload-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156615    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cd8e01d-79f4-43a2-bf55-16dd582372f4" containerName="payload-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156673    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cb57348-cfbe-4f66-af79-7511d3eca1e2" containerName="payload-pre-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156683    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cd8e01d-79f4-43a2-bf55-16dd582372f4" containerName="payload-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156690    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="428f273d-2237-4665-b934-037b999c9fa4" containerName="maas-api"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156696    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="553108f0-a26a-4a43-a0a3-406c8493b4ca" containerName="maas-api"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156702    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="159f0984-3a84-424b-a94b-f477ea325155" containerName="manager"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156708    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d9afa2b-a1ab-4ddb-8d7a-ccfd3feeada4" containerName="payload-pre-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156716    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4ab3d54-066d-44c0-b3a4-f2ef19cf4f5a" containerName="payload-pre-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156720    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="686e2780-ee89-49fe-a8b6-7fab0b539d76" containerName="payload-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156725    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="613f3e0a-d507-45e5-8f3d-2237044c7ffd" containerName="payload-processing"
Sep 16 18:51:17.157109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.156731    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7675640-fd5f-49d1-9711-c4deb19aaa88" containerName="maas-api"
Sep 16 18:51:17.159802 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.159783    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.162422 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.162399    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n9qfx\""
Sep 16 18:51:17.163341 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.163322    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 18:51:17.163440 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.163398    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 18:51:17.163440 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.163414    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 18:51:17.172917 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.172894    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c46454c74-x5p86"]
Sep 16 18:51:17.219708 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.219677    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-webhook-cert\") pod \"maas-controller-6c46454c74-x5p86\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") " pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.219856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.219728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-metrics-certs\") pod \"maas-controller-6c46454c74-x5p86\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") " pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.219856 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.219774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm2p2\" (UniqueName: \"kubernetes.io/projected/49d79755-5868-4629-9efa-827f42eb8e9a-kube-api-access-gm2p2\") pod \"maas-controller-6c46454c74-x5p86\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") " pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.320924 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.320895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-metrics-certs\") pod \"maas-controller-6c46454c74-x5p86\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") " pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.321091 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.320934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm2p2\" (UniqueName: \"kubernetes.io/projected/49d79755-5868-4629-9efa-827f42eb8e9a-kube-api-access-gm2p2\") pod \"maas-controller-6c46454c74-x5p86\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") " pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.321091 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.321040    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-webhook-cert\") pod \"maas-controller-6c46454c74-x5p86\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") " pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.323426 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.323395    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-metrics-certs\") pod \"maas-controller-6c46454c74-x5p86\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") " pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.323426 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.323421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-webhook-cert\") pod \"maas-controller-6c46454c74-x5p86\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") " pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.329979 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.329955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm2p2\" (UniqueName: \"kubernetes.io/projected/49d79755-5868-4629-9efa-827f42eb8e9a-kube-api-access-gm2p2\") pod \"maas-controller-6c46454c74-x5p86\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") " pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.470362 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.470266    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:17.756498 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:17.756420    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c46454c74-x5p86"]
Sep 16 18:51:17.760965 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:51:17.760936    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d79755_5868_4629_9efa_827f42eb8e9a.slice/crio-8a2b0d53903d6994d999615f5e2fd00cf3f816e1c7c320c89f6ff9c6b1523ecc WatchSource:0}: Error finding container 8a2b0d53903d6994d999615f5e2fd00cf3f816e1c7c320c89f6ff9c6b1523ecc: Status 404 returned error can't find the container with id 8a2b0d53903d6994d999615f5e2fd00cf3f816e1c7c320c89f6ff9c6b1523ecc
Sep 16 18:51:18.613860 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:18.613765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-x5p86" event={"ID":"49d79755-5868-4629-9efa-827f42eb8e9a","Type":"ContainerStarted","Data":"f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670"}
Sep 16 18:51:18.613860 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:18.613829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-x5p86" event={"ID":"49d79755-5868-4629-9efa-827f42eb8e9a","Type":"ContainerStarted","Data":"8a2b0d53903d6994d999615f5e2fd00cf3f816e1c7c320c89f6ff9c6b1523ecc"}
Sep 16 18:51:18.614321 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:18.613860    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:18.631830 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:18.631769    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c46454c74-x5p86" podStartSLOduration=1.075110051 podStartE2EDuration="1.631747926s" totalImagesPullingTime="556.637875ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:18.630173786 +0000 UTC m=+2102.186467098" watchObservedRunningTime="2026-09-16 18:51:18.631747926 +0000 UTC m=+2102.188041238"
Sep 16 18:51:29.621059 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:29.621030    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:41.136257 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.136226    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c46454c74-x5p86"]
Sep 16 18:51:41.136721 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.136413    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c46454c74-x5p86" podUID="49d79755-5868-4629-9efa-827f42eb8e9a" containerName="manager" containerID="cri-o://f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670" gracePeriod=30
Sep 16 18:51:41.386166 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.386142    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:41.534851 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.534823    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-metrics-certs\") pod \"49d79755-5868-4629-9efa-827f42eb8e9a\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") "
Sep 16 18:51:41.535014 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.534861    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-webhook-cert\") pod \"49d79755-5868-4629-9efa-827f42eb8e9a\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") "
Sep 16 18:51:41.535014 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.534899    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm2p2\" (UniqueName: \"kubernetes.io/projected/49d79755-5868-4629-9efa-827f42eb8e9a-kube-api-access-gm2p2\") pod \"49d79755-5868-4629-9efa-827f42eb8e9a\" (UID: \"49d79755-5868-4629-9efa-827f42eb8e9a\") "
Sep 16 18:51:41.536852 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.536818    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "49d79755-5868-4629-9efa-827f42eb8e9a" (UID: "49d79755-5868-4629-9efa-827f42eb8e9a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:41.537007 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.536988    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "49d79755-5868-4629-9efa-827f42eb8e9a" (UID: "49d79755-5868-4629-9efa-827f42eb8e9a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:51:41.537088 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.537068    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d79755-5868-4629-9efa-827f42eb8e9a-kube-api-access-gm2p2" (OuterVolumeSpecName: "kube-api-access-gm2p2") pod "49d79755-5868-4629-9efa-827f42eb8e9a" (UID: "49d79755-5868-4629-9efa-827f42eb8e9a"). InnerVolumeSpecName "kube-api-access-gm2p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:51:41.636268 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.636237    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:51:41.636268 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.636263    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49d79755-5868-4629-9efa-827f42eb8e9a-webhook-cert\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:51:41.636424 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.636275    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm2p2\" (UniqueName: \"kubernetes.io/projected/49d79755-5868-4629-9efa-827f42eb8e9a-kube-api-access-gm2p2\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:51:41.727288 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.727258    2576 generic.go:356] "Generic (PLEG): container finished" podID="49d79755-5868-4629-9efa-827f42eb8e9a" containerID="f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670" exitCode=0
Sep 16 18:51:41.727435 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.727299    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-x5p86"
Sep 16 18:51:41.727435 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.727342    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-x5p86" event={"ID":"49d79755-5868-4629-9efa-827f42eb8e9a","Type":"ContainerDied","Data":"f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670"}
Sep 16 18:51:41.727435 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.727399    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-x5p86" event={"ID":"49d79755-5868-4629-9efa-827f42eb8e9a","Type":"ContainerDied","Data":"8a2b0d53903d6994d999615f5e2fd00cf3f816e1c7c320c89f6ff9c6b1523ecc"}
Sep 16 18:51:41.727435 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.727426    2576 scope.go:117] "RemoveContainer" containerID="f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670"
Sep 16 18:51:41.736999 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.736982    2576 scope.go:117] "RemoveContainer" containerID="f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670"
Sep 16 18:51:41.737266 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:51:41.737244    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670\": container with ID starting with f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670 not found: ID does not exist" containerID="f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670"
Sep 16 18:51:41.737332 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.737277    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670"} err="failed to get container status \"f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670\": rpc error: code = NotFound desc = could not find container \"f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670\": container with ID starting with f01abdad22991f2a8ac36b32ac45b787608669f1c1b4f0a7824dad0430fcb670 not found: ID does not exist"
Sep 16 18:51:41.754015 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.753988    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c46454c74-x5p86"]
Sep 16 18:51:41.762842 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:41.762816    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c46454c74-x5p86"]
Sep 16 18:51:42.640556 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.640526    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c46454c74-bz8kq"]
Sep 16 18:51:42.640944 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.640914    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d79755-5868-4629-9efa-827f42eb8e9a" containerName="manager"
Sep 16 18:51:42.640944 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.640925    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d79755-5868-4629-9efa-827f42eb8e9a" containerName="manager"
Sep 16 18:51:42.641026 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.640992    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d79755-5868-4629-9efa-827f42eb8e9a" containerName="manager"
Sep 16 18:51:42.645492 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.645473    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.651162 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.651130    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 16 18:51:42.651311 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.651130    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-n9qfx\""
Sep 16 18:51:42.651311 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.651144    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 16 18:51:42.651311 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.651148    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 16 18:51:42.667044 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.667024    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c46454c74-bz8kq"]
Sep 16 18:51:42.746042 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.746015    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db67e372-2d73-4a25-9616-091766c7ab6e-webhook-cert\") pod \"maas-controller-6c46454c74-bz8kq\" (UID: \"db67e372-2d73-4a25-9616-091766c7ab6e\") " pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.746177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.746053    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db67e372-2d73-4a25-9616-091766c7ab6e-metrics-certs\") pod \"maas-controller-6c46454c74-bz8kq\" (UID: \"db67e372-2d73-4a25-9616-091766c7ab6e\") " pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.746177 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.746112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jzvf\" (UniqueName: \"kubernetes.io/projected/db67e372-2d73-4a25-9616-091766c7ab6e-kube-api-access-4jzvf\") pod \"maas-controller-6c46454c74-bz8kq\" (UID: \"db67e372-2d73-4a25-9616-091766c7ab6e\") " pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.847636 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.847610    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db67e372-2d73-4a25-9616-091766c7ab6e-webhook-cert\") pod \"maas-controller-6c46454c74-bz8kq\" (UID: \"db67e372-2d73-4a25-9616-091766c7ab6e\") " pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.847757 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.847665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db67e372-2d73-4a25-9616-091766c7ab6e-metrics-certs\") pod \"maas-controller-6c46454c74-bz8kq\" (UID: \"db67e372-2d73-4a25-9616-091766c7ab6e\") " pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.847757 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.847733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jzvf\" (UniqueName: \"kubernetes.io/projected/db67e372-2d73-4a25-9616-091766c7ab6e-kube-api-access-4jzvf\") pod \"maas-controller-6c46454c74-bz8kq\" (UID: \"db67e372-2d73-4a25-9616-091766c7ab6e\") " pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.849969 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.849947    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/db67e372-2d73-4a25-9616-091766c7ab6e-metrics-certs\") pod \"maas-controller-6c46454c74-bz8kq\" (UID: \"db67e372-2d73-4a25-9616-091766c7ab6e\") " pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.850065 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.850001    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/db67e372-2d73-4a25-9616-091766c7ab6e-webhook-cert\") pod \"maas-controller-6c46454c74-bz8kq\" (UID: \"db67e372-2d73-4a25-9616-091766c7ab6e\") " pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.858830 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.858808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jzvf\" (UniqueName: \"kubernetes.io/projected/db67e372-2d73-4a25-9616-091766c7ab6e-kube-api-access-4jzvf\") pod \"maas-controller-6c46454c74-bz8kq\" (UID: \"db67e372-2d73-4a25-9616-091766c7ab6e\") " pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:42.943712 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.943691    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49d79755-5868-4629-9efa-827f42eb8e9a" path="/var/lib/kubelet/pods/49d79755-5868-4629-9efa-827f42eb8e9a/volumes"
Sep 16 18:51:42.954884 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:42.954868    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:43.247689 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:43.247612    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c46454c74-bz8kq"]
Sep 16 18:51:43.250489 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:51:43.250463    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb67e372_2d73_4a25_9616_091766c7ab6e.slice/crio-cf6ef6bbf5de741466627cf5a44bdeecd80f4bb310d9b736c7d4eab94a5b591c WatchSource:0}: Error finding container cf6ef6bbf5de741466627cf5a44bdeecd80f4bb310d9b736c7d4eab94a5b591c: Status 404 returned error can't find the container with id cf6ef6bbf5de741466627cf5a44bdeecd80f4bb310d9b736c7d4eab94a5b591c
Sep 16 18:51:43.739733 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:43.739704    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-bz8kq" event={"ID":"db67e372-2d73-4a25-9616-091766c7ab6e","Type":"ContainerStarted","Data":"cf6ef6bbf5de741466627cf5a44bdeecd80f4bb310d9b736c7d4eab94a5b591c"}
Sep 16 18:51:44.745981 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:44.745946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c46454c74-bz8kq" event={"ID":"db67e372-2d73-4a25-9616-091766c7ab6e","Type":"ContainerStarted","Data":"af3a0e54f70d6855157e1f852f38bd261898d5a7d7a0efa44f5b950a0b591539"}
Sep 16 18:51:44.746397 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:44.746077    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:51:44.773039 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:44.772997    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c46454c74-bz8kq" podStartSLOduration=2.221463881 podStartE2EDuration="2.772981641s" totalImagesPullingTime="551.51776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:44.772868006 +0000 UTC m=+2128.329161318" watchObservedRunningTime="2026-09-16 18:51:44.772981641 +0000 UTC m=+2128.329275005"
Sep 16 18:51:55.753071 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:51:55.753039    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c46454c74-bz8kq"
Sep 16 18:55:59.931421 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:55:59.931387    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"]
Sep 16 18:55:59.934883 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:55:59.934857    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:55:59.938226 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:55:59.938183    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-r78k9\""
Sep 16 18:55:59.938336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:55:59.938220    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 16 18:55:59.938336 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:55:59.938300    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 16 18:55:59.941918 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:55:59.941894    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-75c7469999-zpccj"]
Sep 16 18:55:59.945726 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:55:59.945709    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:55:59.951126 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:55:59.951085    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"]
Sep 16 18:55:59.957376 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:55:59.957277    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75c7469999-zpccj"]
Sep 16 18:56:00.017751 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.017722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-metrics-certs\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.017967 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.017767    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-metrics-certs\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.017967 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.017785    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-maas-api-tls\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.017967 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.017873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-maas-api-tls\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.017967 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.017913    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnv9v\" (UniqueName: \"kubernetes.io/projected/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-kube-api-access-jnv9v\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.017967 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.017952    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-openshift-service-ca\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.018261 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.018013    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6x6x\" (UniqueName: \"kubernetes.io/projected/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-kube-api-access-n6x6x\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.018261 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.018043    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-openshift-service-ca\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.119012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.118982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6x6x\" (UniqueName: \"kubernetes.io/projected/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-kube-api-access-n6x6x\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.119012 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.119019    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-openshift-service-ca\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.119333 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.119071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-metrics-certs\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.119333 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.119104    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-metrics-certs\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.119333 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.119121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-maas-api-tls\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.119333 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.119155    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-maas-api-tls\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.119333 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.119189    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnv9v\" (UniqueName: \"kubernetes.io/projected/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-kube-api-access-jnv9v\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.119333 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.119249    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-openshift-service-ca\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.119962 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.119939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-openshift-service-ca\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.120057 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.119939    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-openshift-service-ca\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.122373 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.122341    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-metrics-certs\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.122477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.122374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-maas-api-tls\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.122477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.122381    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-maas-api-tls\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.122477 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.122407    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-metrics-certs\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.126681 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.126657    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6x6x\" (UniqueName: \"kubernetes.io/projected/f7101d39-51ce-45af-9aad-9e5ccf6aa7f4-kube-api-access-n6x6x\") pod \"maas-api-75c7469999-zpccj\" (UID: \"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4\") " pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.127403 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.127387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnv9v\" (UniqueName: \"kubernetes.io/projected/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-kube-api-access-jnv9v\") pod \"maas-api-6ddb7ffb7f-ggqmx\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") " pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.246973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.246901    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:00.259282 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.259103    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:00.523366 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.523337    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"]
Sep 16 18:56:00.525487 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:56:00.525453    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946aa8bd_bca0_4e01_8cc1_2ff3a041eead.slice/crio-933bad6251cb06e3b88a72a7c929426761a88115903e069a1a2985b8de1d0456 WatchSource:0}: Error finding container 933bad6251cb06e3b88a72a7c929426761a88115903e069a1a2985b8de1d0456: Status 404 returned error can't find the container with id 933bad6251cb06e3b88a72a7c929426761a88115903e069a1a2985b8de1d0456
Sep 16 18:56:00.526757 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.526740    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 18:56:00.545795 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.545776    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75c7469999-zpccj"]
Sep 16 18:56:00.548851 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:56:00.548828    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7101d39_51ce_45af_9aad_9e5ccf6aa7f4.slice/crio-ec57d6a975c997b1d97c3609a5a8e84da0c824717bf7642e7ab508fdcc829614 WatchSource:0}: Error finding container ec57d6a975c997b1d97c3609a5a8e84da0c824717bf7642e7ab508fdcc829614: Status 404 returned error can't find the container with id ec57d6a975c997b1d97c3609a5a8e84da0c824717bf7642e7ab508fdcc829614
Sep 16 18:56:00.990280 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.990245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx" event={"ID":"946aa8bd-bca0-4e01-8cc1-2ff3a041eead","Type":"ContainerStarted","Data":"933bad6251cb06e3b88a72a7c929426761a88115903e069a1a2985b8de1d0456"}
Sep 16 18:56:00.991801 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:00.991774    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj" event={"ID":"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4","Type":"ContainerStarted","Data":"ec57d6a975c997b1d97c3609a5a8e84da0c824717bf7642e7ab508fdcc829614"}
Sep 16 18:56:01.998923 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:01.998882    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx" event={"ID":"946aa8bd-bca0-4e01-8cc1-2ff3a041eead","Type":"ContainerStarted","Data":"6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395"}
Sep 16 18:56:01.999450 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:01.998949    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:02.000413 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:02.000393    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj" event={"ID":"f7101d39-51ce-45af-9aad-9e5ccf6aa7f4","Type":"ContainerStarted","Data":"04a110bc28765c61498bcd0d1f32ff2a6b009bd9f85aab4b4f5492d911bf339e"}
Sep 16 18:56:02.000537 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:02.000520    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:02.019536 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:02.019491    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx" podStartSLOduration=2.550341863 podStartE2EDuration="3.01947774s" totalImagesPullingTime="469.135877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:55:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:56:02.018728966 +0000 UTC m=+2385.575022280" watchObservedRunningTime="2026-09-16 18:56:02.01947774 +0000 UTC m=+2385.575771051"
Sep 16 18:56:02.038535 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:02.038489    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj" podStartSLOduration=2.643627734 podStartE2EDuration="3.038474806s" totalImagesPullingTime="394.847072ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:55:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:56:02.036876246 +0000 UTC m=+2385.593169557" watchObservedRunningTime="2026-09-16 18:56:02.038474806 +0000 UTC m=+2385.594768116"
Sep 16 18:56:08.011787 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.011754    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:08.012157 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.011828    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-75c7469999-zpccj"
Sep 16 18:56:08.091751 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.091716    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"]
Sep 16 18:56:08.091931 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.091897    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx" podUID="946aa8bd-bca0-4e01-8cc1-2ff3a041eead" containerName="maas-api" containerID="cri-o://6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395" gracePeriod=30
Sep 16 18:56:08.340455 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.340431    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:08.497780 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.497742    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-openshift-service-ca\") pod \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") "
Sep 16 18:56:08.497973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.497856    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-metrics-certs\") pod \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") "
Sep 16 18:56:08.497973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.497896    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnv9v\" (UniqueName: \"kubernetes.io/projected/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-kube-api-access-jnv9v\") pod \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") "
Sep 16 18:56:08.497973 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.497925    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-maas-api-tls\") pod \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\" (UID: \"946aa8bd-bca0-4e01-8cc1-2ff3a041eead\") "
Sep 16 18:56:08.498163 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.498129    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "946aa8bd-bca0-4e01-8cc1-2ff3a041eead" (UID: "946aa8bd-bca0-4e01-8cc1-2ff3a041eead"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 18:56:08.500113 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.500088    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-kube-api-access-jnv9v" (OuterVolumeSpecName: "kube-api-access-jnv9v") pod "946aa8bd-bca0-4e01-8cc1-2ff3a041eead" (UID: "946aa8bd-bca0-4e01-8cc1-2ff3a041eead"). InnerVolumeSpecName "kube-api-access-jnv9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 18:56:08.500385 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.500366    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "946aa8bd-bca0-4e01-8cc1-2ff3a041eead" (UID: "946aa8bd-bca0-4e01-8cc1-2ff3a041eead"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:56:08.500465 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.500442    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "946aa8bd-bca0-4e01-8cc1-2ff3a041eead" (UID: "946aa8bd-bca0-4e01-8cc1-2ff3a041eead"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 18:56:08.599618 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.599553    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-metrics-certs\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:56:08.599618 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.599578    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnv9v\" (UniqueName: \"kubernetes.io/projected/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-kube-api-access-jnv9v\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:56:08.599618 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.599589    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-maas-api-tls\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:56:08.599618 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:08.599599    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/946aa8bd-bca0-4e01-8cc1-2ff3a041eead-openshift-service-ca\") on node \"ip-10-0-129-142.ec2.internal\" DevicePath \"\""
Sep 16 18:56:09.034562 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:09.034526    2576 generic.go:356] "Generic (PLEG): container finished" podID="946aa8bd-bca0-4e01-8cc1-2ff3a041eead" containerID="6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395" exitCode=0
Sep 16 18:56:09.035021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:09.034566    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"
Sep 16 18:56:09.035021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:09.034598    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx" event={"ID":"946aa8bd-bca0-4e01-8cc1-2ff3a041eead","Type":"ContainerDied","Data":"6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395"}
Sep 16 18:56:09.035021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:09.034646    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx" event={"ID":"946aa8bd-bca0-4e01-8cc1-2ff3a041eead","Type":"ContainerDied","Data":"933bad6251cb06e3b88a72a7c929426761a88115903e069a1a2985b8de1d0456"}
Sep 16 18:56:09.035021 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:09.034667    2576 scope.go:117] "RemoveContainer" containerID="6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395"
Sep 16 18:56:09.043670 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:09.043655    2576 scope.go:117] "RemoveContainer" containerID="6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395"
Sep 16 18:56:09.043918 ip-10-0-129-142 kubenswrapper[2576]: E0916 18:56:09.043897    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395\": container with ID starting with 6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395 not found: ID does not exist" containerID="6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395"
Sep 16 18:56:09.043967 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:09.043927    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395"} err="failed to get container status \"6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395\": rpc error: code = NotFound desc = could not find container \"6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395\": container with ID starting with 6e8e439b6e4966584f2d2280a947d4325ac686dc5d8cd9be9d9aea217155c395 not found: ID does not exist"
Sep 16 18:56:09.051700 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:09.051680    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"]
Sep 16 18:56:09.054588 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:09.054570    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6ddb7ffb7f-ggqmx"]
Sep 16 18:56:10.944534 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:10.944505    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946aa8bd-bca0-4e01-8cc1-2ff3a041eead" path="/var/lib/kubelet/pods/946aa8bd-bca0-4e01-8cc1-2ff3a041eead/volumes"
Sep 16 18:56:35.617180 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:35.617152    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c46454c74-bz8kq_db67e372-2d73-4a25-9616-091766c7ab6e/manager/0.log"
Sep 16 18:56:36.001511 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:36.001471    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz_56ecd90a-dc19-4d0a-aec4-edf4e7049414/copy-manifests/0.log"
Sep 16 18:56:36.008725 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:36.008695    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz_56ecd90a-dc19-4d0a-aec4-edf4e7049414/manager/0.log"
Sep 16 18:56:36.830600 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:36.830569    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-75c7469999-zpccj_f7101d39-51ce-45af-9aad-9e5ccf6aa7f4/maas-api/0.log"
Sep 16 18:56:37.064458 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:37.064428    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-lx6ks_e4500109-c468-4163-a946-d0ce59c9de06/postgres/0.log"
Sep 16 18:56:38.247130 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:38.247101    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-6qbxp_a4bb689b-f7a2-4982-81cd-371f2cfc6823/kuadrant-console-plugin/0.log"
Sep 16 18:56:39.365055 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:39.364976    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79f4c98848-jnvlv_4c283f1b-8108-44fe-8e78-2e5c3fa572fe/kube-auth-proxy/0.log"
Sep 16 18:56:39.746563 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:39.746532    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vr6p8_d1aad725-09fe-4095-ab39-57f3aad985f1/payload-processing/0.log"
Sep 16 18:56:40.223010 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:40.222984    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-4jmpc_a3e1f211-7bf4-4f72-b474-464563ce15c3/main/0.log"
Sep 16 18:56:40.461542 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:40.461511    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-pthsf_bb3ae5ca-1cd1-4668-967d-633957149360/main/0.log"
Sep 16 18:56:40.700471 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:40.700444    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fq9h5m_10d1faf7-a952-473d-bb0c-f5b3c16b53fa/main/0.log"
Sep 16 18:56:49.866893 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:49.866860    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dfxvz_a6598da7-8e1c-498e-be9f-ed44f6b8a43a/global-pull-secret-syncer/0.log"
Sep 16 18:56:50.070475 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:50.070441    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hs5np_ad78a825-31ef-4ae5-9f3a-b1b3031922e5/konnectivity-agent/0.log"
Sep 16 18:56:50.140525 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:50.140446    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-142.ec2.internal_bca848baac81a5ff45bd401fe573fb0f/haproxy/0.log"
Sep 16 18:56:54.968402 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:54.968375    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-6qbxp_a4bb689b-f7a2-4982-81cd-371f2cfc6823/kuadrant-console-plugin/0.log"
Sep 16 18:56:56.604544 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.604514    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1df6ea65-4988-497f-9e20-a65d19771b12/alertmanager/0.log"
Sep 16 18:56:56.627320 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.627295    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1df6ea65-4988-497f-9e20-a65d19771b12/config-reloader/0.log"
Sep 16 18:56:56.649476 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.649451    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1df6ea65-4988-497f-9e20-a65d19771b12/kube-rbac-proxy-web/0.log"
Sep 16 18:56:56.673275 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.673248    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1df6ea65-4988-497f-9e20-a65d19771b12/kube-rbac-proxy/0.log"
Sep 16 18:56:56.700585 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.700565    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1df6ea65-4988-497f-9e20-a65d19771b12/kube-rbac-proxy-metric/0.log"
Sep 16 18:56:56.724596 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.724578    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1df6ea65-4988-497f-9e20-a65d19771b12/prom-label-proxy/0.log"
Sep 16 18:56:56.750020 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.750000    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1df6ea65-4988-497f-9e20-a65d19771b12/init-config-reloader/0.log"
Sep 16 18:56:56.829131 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.829111    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-lcj9x_ea1180ad-4b65-415c-973d-cd8763826bf8/kube-state-metrics/0.log"
Sep 16 18:56:56.864202 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.864141    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-lcj9x_ea1180ad-4b65-415c-973d-cd8763826bf8/kube-rbac-proxy-main/0.log"
Sep 16 18:56:56.897639 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.897617    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-59fbdd77d-lcj9x_ea1180ad-4b65-415c-973d-cd8763826bf8/kube-rbac-proxy-self/0.log"
Sep 16 18:56:56.928041 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.928019    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6876bb786b-5ssjl_8215605a-bdbe-4b67-9c34-2fe8e8997c0d/metrics-server/0.log"
Sep 16 18:56:56.954066 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.954043    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-r67dw_c655ace3-27d3-4eab-9927-1711673bf890/monitoring-plugin/0.log"
Sep 16 18:56:56.988941 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:56.988921    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fjwrq_40510277-75cd-4c48-b079-b0408beb038c/node-exporter/0.log"
Sep 16 18:56:57.008769 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.008730    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fjwrq_40510277-75cd-4c48-b079-b0408beb038c/kube-rbac-proxy/0.log"
Sep 16 18:56:57.037160 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.037141    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fjwrq_40510277-75cd-4c48-b079-b0408beb038c/init-textfile/0.log"
Sep 16 18:56:57.268473 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.268452    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-6wbz6_e8985fab-6493-48bf-ab68-366466f85447/kube-rbac-proxy-main/0.log"
Sep 16 18:56:57.302054 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.302012    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-6wbz6_e8985fab-6493-48bf-ab68-366466f85447/kube-rbac-proxy-self/0.log"
Sep 16 18:56:57.333022 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.333002    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-6wbz6_e8985fab-6493-48bf-ab68-366466f85447/openshift-state-metrics/0.log"
Sep 16 18:56:57.390111 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.390089    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66ff755f-26dd-4520-9f52-da84d1ab1e0a/prometheus/0.log"
Sep 16 18:56:57.426349 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.426325    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66ff755f-26dd-4520-9f52-da84d1ab1e0a/config-reloader/0.log"
Sep 16 18:56:57.451093 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.451071    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66ff755f-26dd-4520-9f52-da84d1ab1e0a/thanos-sidecar/0.log"
Sep 16 18:56:57.483936 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.483882    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66ff755f-26dd-4520-9f52-da84d1ab1e0a/kube-rbac-proxy-web/0.log"
Sep 16 18:56:57.512747 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.512729    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66ff755f-26dd-4520-9f52-da84d1ab1e0a/kube-rbac-proxy/0.log"
Sep 16 18:56:57.537290 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.537266    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66ff755f-26dd-4520-9f52-da84d1ab1e0a/kube-rbac-proxy-thanos/0.log"
Sep 16 18:56:57.560413 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.560393    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66ff755f-26dd-4520-9f52-da84d1ab1e0a/init-config-reloader/0.log"
Sep 16 18:56:57.596596 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.596576    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-rslvr_c90e00fc-eb64-424a-8431-eeb5b9a59914/prometheus-operator/0.log"
Sep 16 18:56:57.617016 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.616996    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-rslvr_c90e00fc-eb64-424a-8431-eeb5b9a59914/kube-rbac-proxy/0.log"
Sep 16 18:56:57.689529 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.689504    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74bcd5c6cf-vqwt2_12b641af-7985-4862-8e5e-35db0bd27931/telemeter-client/0.log"
Sep 16 18:56:57.712770 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.712749    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74bcd5c6cf-vqwt2_12b641af-7985-4862-8e5e-35db0bd27931/reload/0.log"
Sep 16 18:56:57.742145 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.742078    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74bcd5c6cf-vqwt2_12b641af-7985-4862-8e5e-35db0bd27931/kube-rbac-proxy/0.log"
Sep 16 18:56:57.772528 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.772508    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd495f58-jv5bv_60a0d6a7-73b4-4c55-b257-e8549c9ac37c/thanos-query/0.log"
Sep 16 18:56:57.793653 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.793633    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd495f58-jv5bv_60a0d6a7-73b4-4c55-b257-e8549c9ac37c/kube-rbac-proxy-web/0.log"
Sep 16 18:56:57.818469 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.818452    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd495f58-jv5bv_60a0d6a7-73b4-4c55-b257-e8549c9ac37c/kube-rbac-proxy/0.log"
Sep 16 18:56:57.835998 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.835983    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd495f58-jv5bv_60a0d6a7-73b4-4c55-b257-e8549c9ac37c/prom-label-proxy/0.log"
Sep 16 18:56:57.858371 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.858350    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd495f58-jv5bv_60a0d6a7-73b4-4c55-b257-e8549c9ac37c/kube-rbac-proxy-rules/0.log"
Sep 16 18:56:57.879609 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:56:57.879592    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5bfd495f58-jv5bv_60a0d6a7-73b4-4c55-b257-e8549c9ac37c/kube-rbac-proxy-metrics/0.log"
Sep 16 18:57:00.747361 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.747331    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56cb87b9bf-bjstk_5e9ffea6-8a9d-4315-99f5-648c8b919b07/console/0.log"
Sep 16 18:57:00.808806 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.808777    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"]
Sep 16 18:57:00.809224 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.809200    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946aa8bd-bca0-4e01-8cc1-2ff3a041eead" containerName="maas-api"
Sep 16 18:57:00.809273 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.809225    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="946aa8bd-bca0-4e01-8cc1-2ff3a041eead" containerName="maas-api"
Sep 16 18:57:00.809318 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.809308    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="946aa8bd-bca0-4e01-8cc1-2ff3a041eead" containerName="maas-api"
Sep 16 18:57:00.812243 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.812226    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.816707 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.816690    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pncdd\"/\"kube-root-ca.crt\""
Sep 16 18:57:00.817468 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.817452    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pncdd\"/\"openshift-service-ca.crt\""
Sep 16 18:57:00.817577 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.817481    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pncdd\"/\"default-dockercfg-5668f\""
Sep 16 18:57:00.828079 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.828059    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"]
Sep 16 18:57:00.870902 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.870872    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-podres\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.871013 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.870921    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-proc\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.871013 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.870965    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-sys\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.871013 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.871008    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpzlt\" (UniqueName: \"kubernetes.io/projected/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-kube-api-access-wpzlt\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.871146 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.871091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-lib-modules\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.971860 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.971836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-lib-modules\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.971996 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.971907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-podres\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.971996 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.971949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-proc\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.971996 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.971973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-sys\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.972137 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.971998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-lib-modules\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.972137 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.972009    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpzlt\" (UniqueName: \"kubernetes.io/projected/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-kube-api-access-wpzlt\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.972137 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.972039    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-proc\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.972137 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.972049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-sys\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.972137 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.972063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-podres\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:00.981696 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:00.981669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpzlt\" (UniqueName: \"kubernetes.io/projected/35d8ccf3-2fe5-4909-81a7-5763ce0c5c18-kube-api-access-wpzlt\") pod \"perf-node-gather-daemonset-xlbwg\" (UID: \"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18\") " pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:01.122397 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:01.122327    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:01.393129 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:01.393101    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"]
Sep 16 18:57:01.394826 ip-10-0-129-142 kubenswrapper[2576]: W0916 18:57:01.394798    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod35d8ccf3_2fe5_4909_81a7_5763ce0c5c18.slice/crio-82d36dbd7bdc88ae99061b3de6dfd4373cc5e5274e2e6bdf08893a84999ab087 WatchSource:0}: Error finding container 82d36dbd7bdc88ae99061b3de6dfd4373cc5e5274e2e6bdf08893a84999ab087: Status 404 returned error can't find the container with id 82d36dbd7bdc88ae99061b3de6dfd4373cc5e5274e2e6bdf08893a84999ab087
Sep 16 18:57:02.305889 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:02.305852    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg" event={"ID":"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18","Type":"ContainerStarted","Data":"054f85e17efe84be7cefc307bb4d93bd74b6f7685c83dc1359aaffe4a7d5dfff"}
Sep 16 18:57:02.306298 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:02.305938    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:02.306298 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:02.305955    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg" event={"ID":"35d8ccf3-2fe5-4909-81a7-5763ce0c5c18","Type":"ContainerStarted","Data":"82d36dbd7bdc88ae99061b3de6dfd4373cc5e5274e2e6bdf08893a84999ab087"}
Sep 16 18:57:02.331364 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:02.331310    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg" podStartSLOduration=2.331294758 podStartE2EDuration="2.331294758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:57:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:57:02.330381232 +0000 UTC m=+2445.886674542" watchObservedRunningTime="2026-09-16 18:57:02.331294758 +0000 UTC m=+2445.887588068"
Sep 16 18:57:03.037642 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:03.037610    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7kjdd_ab4e4cca-0a08-4227-8195-52c3b543efa3/dns/0.log"
Sep 16 18:57:03.064086 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:03.064065    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7kjdd_ab4e4cca-0a08-4227-8195-52c3b543efa3/kube-rbac-proxy/0.log"
Sep 16 18:57:03.196147 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:03.196123    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wddn6_3f7fa44e-0a12-4f6c-9a7e-a74dab335559/dns-node-resolver/0.log"
Sep 16 18:57:03.928992 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:03.928962    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tsg74_c4b4ff85-1357-423c-b6d1-b7f7181b35ff/node-ca/0.log"
Sep 16 18:57:05.396869 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:05.396838    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79f4c98848-jnvlv_4c283f1b-8108-44fe-8e78-2e5c3fa572fe/kube-auth-proxy/0.log"
Sep 16 18:57:05.499930 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:05.499901    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vr6p8_d1aad725-09fe-4095-ab39-57f3aad985f1/payload-processing/0.log"
Sep 16 18:57:06.510923 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:06.510893    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mcq6w_5445c3f6-e0a4-4389-bdd7-38b1ea61c111/serve-healthcheck-canary/0.log"
Sep 16 18:57:07.277109 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:07.277077    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s779v_1e835b28-5963-49b5-9127-892f33c8ec74/kube-rbac-proxy/0.log"
Sep 16 18:57:07.302172 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:07.302148    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s779v_1e835b28-5963-49b5-9127-892f33c8ec74/exporter/0.log"
Sep 16 18:57:07.342907 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:07.342879    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s779v_1e835b28-5963-49b5-9127-892f33c8ec74/extractor/0.log"
Sep 16 18:57:08.321303 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:08.321276    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pncdd/perf-node-gather-daemonset-xlbwg"
Sep 16 18:57:10.272646 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:10.272620    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c46454c74-bz8kq_db67e372-2d73-4a25-9616-091766c7ab6e/manager/0.log"
Sep 16 18:57:10.420797 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:10.420771    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz_56ecd90a-dc19-4d0a-aec4-edf4e7049414/manager/0.log"
Sep 16 18:57:10.448884 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:10.448864    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cfd5c8dfd-jdwrz_56ecd90a-dc19-4d0a-aec4-edf4e7049414/copy-manifests/0.log"
Sep 16 18:57:11.801537 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:11.801503    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79f4c98848-jnvlv_4c283f1b-8108-44fe-8e78-2e5c3fa572fe/kube-auth-proxy/0.log"
Sep 16 18:57:11.925053 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:11.925026    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vr6p8_d1aad725-09fe-4095-ab39-57f3aad985f1/payload-processing/0.log"
Sep 16 18:57:12.040379 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:12.040354    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5555c8c497-pkp5p_b60c45ef-6b8a-4a85-b11c-dd3534c612ad/manager/0.log"
Sep 16 18:57:12.100166 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:12.100097    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-29cmh_67717b12-c798-4df0-b7a5-41c0c9df16ca/openshift-lws-operator/0.log"
Sep 16 18:57:14.009127 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:14.009090    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-79f4c98848-jnvlv_4c283f1b-8108-44fe-8e78-2e5c3fa572fe/kube-auth-proxy/0.log"
Sep 16 18:57:14.093629 ip-10-0-129-142 kubenswrapper[2576]: I0916 18:57:14.093600    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vr6p8_d1aad725-09fe-4095-ab39-57f3aad985f1/payload-processing/0.log"