Sep 18 19:56:58.056045 ip-10-0-130-248 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 19:57:01.516833 ip-10-0-130-248 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 19:57:01.516833 ip-10-0-130-248 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 19:57:01.516833 ip-10-0-130-248 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 19:57:01.516833 ip-10-0-130-248 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 19:57:01.516833 ip-10-0-130-248 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 19:57:01.517557 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.516904    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 19:57:01.519859 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519842    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 19:57:01.519859 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519858    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519863    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519866    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519869    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519872    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519875    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519877    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519880    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519883    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519886    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519889    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519892    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519894    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519897    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519900    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519907    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519910    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519913    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519915    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519918    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 19:57:01.519923 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519921    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519923    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519926    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519929    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519932    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519934    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519937    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519940    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519942    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519944    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519948    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519950    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519953    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519956    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519958    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519961    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519963    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519966    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519968    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519972    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 19:57:01.520431 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519974    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519977    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519981    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519985    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519988    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519990    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519994    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519997    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.519999    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520002    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520004    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520007    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520009    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520012    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520015    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520018    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520020    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520023    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520025    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520028    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 19:57:01.520936 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520030    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520033    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520035    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520038    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520041    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520043    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520046    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520049    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520051    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520056    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520061    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520064    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520068    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520071    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520073    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520076    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520078    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520081    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520084    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 19:57:01.521416 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520088    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520092    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520095    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520098    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520100    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520103    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520106    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520110    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520113    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520116    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520119    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520122    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520124    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520127    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520129    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520132    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520134    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520137    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520140    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 19:57:01.521889 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520862    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520869    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520873    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520876    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520879    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520883    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520886    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520888    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520891    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520894    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520897    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520900    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520903    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520905    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520908    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520911    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520914    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520917    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520919    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520922    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 19:57:01.522364 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520925    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520927    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520932    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520935    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520939    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520942    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520945    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520948    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520951    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520954    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520956    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520959    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520962    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520965    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520968    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520970    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520973    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520975    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520979    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 19:57:01.522871 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520981    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520984    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520987    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520990    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520992    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520996    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.520999    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521002    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521004    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521007    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521010    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521013    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521016    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521019    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521022    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521024    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521027    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521029    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521032    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521036    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 19:57:01.523331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521040    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521042    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521045    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521048    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521050    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521053    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521055    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521058    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521060    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521063    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521066    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521068    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521071    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521076    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521078    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521082    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521086    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521089    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521092    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 19:57:01.523835 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521095    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521098    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521101    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521104    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521107    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521110    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521113    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521116    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521119    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521121    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521124    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521127    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521130    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521133    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521136    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521138    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521141    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521143    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521146    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 19:57:01.524315 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521149    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521169    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521533    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521539    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521542    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521545    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521548    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521551    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521554    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521556    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521559    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521562    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521565    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521567    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521570    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521573    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521575    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521578    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521581    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521583    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 19:57:01.524808 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521585    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521588    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521591    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521593    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521596    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521598    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521601    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521603    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521606    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521609    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521611    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521614    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521616    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521619    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521621    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521624    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521626    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521629    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521632    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521635    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 19:57:01.525329 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521638    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521641    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521643    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521646    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521648    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521651    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521653    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521656    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521658    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521661    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521664    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521666    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521668    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521671    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521673    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521676    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521678    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521696    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521700    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 19:57:01.525830 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521703    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521707    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521711    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521714    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521716    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521719    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521722    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521724    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521727    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521729    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521732    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521734    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521737    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521740    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521744    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521748    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521755    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521758    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521761    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 19:57:01.526290 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521764    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521767    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521769    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521773    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521775    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521779    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521783    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521786    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521788    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521791    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521794    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521796    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521799    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521802    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521805    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521808    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521810    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521812    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521815    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 19:57:01.526831 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521818    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521820    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521822    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.521825    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521893    2567 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521928    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521938    2567 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521942    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521954    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521957    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521963    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521967    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521970    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521973    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521977    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521981    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521984    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521987    2567 flags.go:64] FLAG: --cgroup-root=""
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521990    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521992    2567 flags.go:64] FLAG: --client-ca-file=""
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521995    2567 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.521998    2567 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522005    2567 flags.go:64] FLAG: --cluster-domain=""
Sep 18 19:57:01.527297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522007    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522010    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522015    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522019    2567 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522023    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522026    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522029    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522032    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522035    2567 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522038    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522041    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522044    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522047    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522051    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522055    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522058    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522060    2567 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522063    2567 flags.go:64] FLAG: --enable-server="true"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522067    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522074    2567 flags.go:64] FLAG: --event-burst="100"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522077    2567 flags.go:64] FLAG: --event-qps="50"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522080    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522083    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522085    2567 flags.go:64] FLAG: --eviction-hard=""
Sep 18 19:57:01.527857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522089    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522092    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522095    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522098    2567 flags.go:64] FLAG: --eviction-soft=""
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522101    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522104    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522107    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522110    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522113    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522115    2567 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522120    2567 flags.go:64] FLAG: --feature-gates=""
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522123    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522126    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522130    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522133    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522136    2567 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522139    2567 flags.go:64] FLAG: --help="false"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522142    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522145    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522148    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522151    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522154    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522158    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522161    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 19:57:01.528435 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522163    2567 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522167    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522169    2567 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522174    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522177    2567 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522180    2567 flags.go:64] FLAG: --kube-reserved=""
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522183    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522186    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522189    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522192    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522194    2567 flags.go:64] FLAG: --lock-file=""
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522197    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522200    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522203    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522208    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522211    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522214    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522217    2567 flags.go:64] FLAG: --logging-format="text"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522221    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522224    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522227    2567 flags.go:64] FLAG: --manifest-url=""
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522230    2567 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522234    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522237    2567 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522241    2567 flags.go:64] FLAG: --max-pods="110"
Sep 18 19:57:01.529074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522244    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522247    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522250    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522253    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522256    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522258    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522261    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522269    2567 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522272    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522275    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522279    2567 flags.go:64] FLAG: --pod-cidr=""
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522283    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522289    2567 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522292    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522295    2567 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522298    2567 flags.go:64] FLAG: --port="10250"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522301    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522304    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aebf219e076ad718"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522307    2567 flags.go:64] FLAG: --qos-reserved=""
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522310    2567 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522312    2567 flags.go:64] FLAG: --register-node="true"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522315    2567 flags.go:64] FLAG: --register-with-taints=""
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522319    2567 flags.go:64] FLAG: --registry-burst="10"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522322    2567 flags.go:64] FLAG: --registry-qps="5"
Sep 18 19:57:01.529699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522324    2567 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522332    2567 flags.go:64] FLAG: --reserved-memory=""
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522335    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522339    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522342    2567 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522345    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522348    2567 flags.go:64] FLAG: --runonce="false"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522350    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522354    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522357    2567 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522360    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522362    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522366    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522369    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522372    2567 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522376    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522378    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522381    2567 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522384    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522388    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522391    2567 flags.go:64] FLAG: --system-cgroups=""
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522394    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522400    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522403    2567 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522406    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 19:57:01.530313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522410    2567 flags.go:64] FLAG: --tls-min-version=""
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522413    2567 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522416    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522419    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522422    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522424    2567 flags.go:64] FLAG: --v="2"
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522428    2567 flags.go:64] FLAG: --version="false"
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522435    2567 flags.go:64] FLAG: --vmodule=""
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522440    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.522444    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522563    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522568    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522572    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522575    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522578    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522581    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522584    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522587    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522589    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522592    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522595    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 19:57:01.530932 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522598    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522601    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522604    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522606    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522609    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522613    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522616    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522619    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522622    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522625    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522628    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522631    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522633    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522636    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522639    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522641    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522644    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522646    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522649    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522653    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 19:57:01.531439 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522656    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522659    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522661    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522664    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522667    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522670    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522672    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522675    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522678    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522695    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522698    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522700    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522703    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522706    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522708    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522712    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522714    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522718    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522721    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522724    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 19:57:01.532052 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522727    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522730    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522733    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522735    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522738    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522741    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522743    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522746    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522749    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522751    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522754    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522757    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522760    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522763    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522766    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522768    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522771    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522774    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522776    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522779    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 19:57:01.532850 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522782    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522784    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522787    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522789    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522792    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522795    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522797    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522800    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522803    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522806    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522809    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522812    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522814    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522818    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522821    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522825    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522828    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522831    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522834    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522837    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 19:57:01.533407 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522840    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522842    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522845    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522849    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522851    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522854    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522857    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.522859    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.523623    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.530498    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.530513    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.530533    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.530539    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530594    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530598    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 19:57:01.533912 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530602    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530605    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530609    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530611    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530614    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530617    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530620    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530623    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530626    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530629    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530631    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530635    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530637    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530640    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530643    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530645    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530648    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530651    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530653    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 19:57:01.534335 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530656    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530659    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530662    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530665    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530667    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530670    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530673    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530676    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530679    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530694    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530698    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530701    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530703    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530706    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530708    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530713    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530716    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530720    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530722    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530725    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 19:57:01.534848 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530728    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530731    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530734    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530737    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530740    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530743    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530746    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530748    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530751    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530754    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530756    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530759    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530762    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530764    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530768    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530771    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530774    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530777    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530779    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530782    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 19:57:01.535331 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530784    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530787    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530790    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530792    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530795    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530798    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530800    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530803    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530806    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530808    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530811    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530813    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530816    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530819    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530821    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530824    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530826    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530829    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530832    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530835    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 19:57:01.535890 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530837    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530840    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530842    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530845    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530849    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530852    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530856    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530858    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530861    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530864    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530866    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530869    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530871    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530874    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530877    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530880    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530883    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 19:57:01.536382 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.530885    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.530890    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531018    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531024    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531028    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531031    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531034    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531037    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531040    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531043    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531045    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531048    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531051    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531054    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531057    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 19:57:01.536806 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531060    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531063    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531066    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531069    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531071    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531074    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531076    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531079    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531082    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531084    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531087    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531089    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531092    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531094    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531097    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531102    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531105    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531108    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531111    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531113    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 19:57:01.537173 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531116    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531119    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531121    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531124    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531127    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531129    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531132    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531135    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531137    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531140    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531142    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531145    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531147    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531150    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531152    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531155    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531157    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531160    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531162    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 19:57:01.537861 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531165    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531167    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531171    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531174    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531177    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531180    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531182    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531185    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531188    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531191    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531195    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531198    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531200    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531203    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531206    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531208    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531211    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531214    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531216    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 19:57:01.538341 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531219    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531222    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531226    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531229    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531232    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531234    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531237    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531240    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531242    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531245    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531247    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531250    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531253    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531255    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531258    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531260    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531263    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531265    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531269    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531272    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 19:57:01.538829 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531274    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531277    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531280    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531284    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531287    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531290    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531292    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:01.531295    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.531300    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.531412    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.534229    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.535193    2567 server.go:1019] "Starting client certificate rotation"
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.535292    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 19:57:01.539320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.535321    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 19:57:01.539759 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.537698    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 19:57:01.539814 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.539795    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 19:57:01.543556 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.543542    2567 log.go:25] "Validated CRI v1 runtime API"
Sep 18 19:57:01.544953 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.544938    2567 log.go:25] "Validated CRI v1 image API"
Sep 18 19:57:01.545638 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.545626    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 19:57:01.547829 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.547811    2567 fs.go:135] Filesystem UUIDs: map[73bcafba-a7e4-4829-a8e3-e0cead2355a0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cd10b203-4d64-44b6-9a4c-b77cc9b9d71d:/dev/nvme0n1p3]
Sep 18 19:57:01.547879 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.547830    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 19:57:01.552261 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.552155    2567 manager.go:216] Machine: {Timestamp:2026-09-18 19:57:01.551664055 +0000 UTC m=+2.112247502 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100183 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ee4815167cf4ceba98a29c4b8fc49 SystemUUID:ec2ee481-5167-cf4c-eba9-8a29c4b8fc49 BootID:bf130fb0-d4ce-4148-b99c-ea251328c552 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d2:b1:bb:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d2:b1:bb:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:32:a9:7e:89:53 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 19:57:01.552261 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.552256    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 19:57:01.552394 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.552330    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 19:57:01.554326 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.554301    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 19:57:01.554461 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.554330    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-248.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 19:57:01.555488 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.555477    2567 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 19:57:01.555523 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.555492    2567 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 19:57:01.555523 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.555500    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 19:57:01.555523 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.555511    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 19:57:01.555616 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.555582    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 19:57:01.559232 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.559219    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 19:57:01.559350 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.559340    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 19:57:01.559391 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.559376    2567 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 19:57:01.559391 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.559387    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 19:57:01.559461 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.559397    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 19:57:01.559461 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.559406    2567 kubelet.go:398] "Adding apiserver pod source"
Sep 18 19:57:01.559461 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.559414    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 19:57:01.559788 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.559776    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 19:57:01.559831 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.559795    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 19:57:01.562116 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.562098    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 19:57:01.564171 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.564151    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 19:57:01.564249 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.564182    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 19:57:01.568449 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568425    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 19:57:01.568538 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568458    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 19:57:01.568538 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568468    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 19:57:01.568538 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568479    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 19:57:01.568538 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568504    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 19:57:01.568538 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568515    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 19:57:01.568538 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568537    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 19:57:01.568788 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568548    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 19:57:01.568788 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568560    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 19:57:01.568788 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568571    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 19:57:01.568788 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568609    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 19:57:01.568788 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568621    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 19:57:01.568788 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568675    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 19:57:01.568788 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.568700    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 19:57:01.572545 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.572524    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 19:57:01.574925 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.574910    2567 server.go:1268] "Started kubelet"
Sep 18 19:57:01.575090 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.575050    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 19:57:01.575150 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.575135    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-248.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 19:57:01.575196 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.575142    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 19:57:01.575196 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.575157    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 19:57:01.575851 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.575823    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 19:57:01.575843 ip-10-0-130-248 systemd[1]: Started Kubernetes Kubelet.
Sep 18 19:57:01.576463 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.575971    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 19:57:01.576463 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.576235    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 19:57:01.576463 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.576337    2567 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 19:57:01.581427 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.581401    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 19:57:01.583943 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.583929    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 19:57:01.584009 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.583939    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 19:57:01.584059 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.584009    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:01.584510 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.584494    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 19:57:01.584577 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.584529    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 19:57:01.584577 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.584503    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 19:57:01.584701 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.584629    2567 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 19:57:01.584701 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.584636    2567 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 19:57:01.584945 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.584922    2567 factory.go:55] Registering systemd factory
Sep 18 19:57:01.585001 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.584953    2567 factory.go:223] Registration of the systemd container factory successfully
Sep 18 19:57:01.585174 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.585162    2567 factory.go:153] Registering CRI-O factory
Sep 18 19:57:01.585216 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.585181    2567 factory.go:223] Registration of the crio container factory successfully
Sep 18 19:57:01.585247 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.585234    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 19:57:01.585297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.585254    2567 factory.go:103] Registering Raw factory
Sep 18 19:57:01.585297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.585271    2567 manager.go:1196] Started watching for new ooms in manager
Sep 18 19:57:01.585849 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.585830    2567 manager.go:349] Starting recovery of all containers
Sep 18 19:57:01.585956 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.585933    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 19:57:01.586436 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.586395    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 19:57:01.594540 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.594511    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l6l75"
Sep 18 19:57:01.596191 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.596176    2567 manager.go:354] Recovery completed
Sep 18 19:57:01.606437 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.606402    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l6l75"
Sep 18 19:57:01.613923 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.613905    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 19:57:01.615645 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.615630    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 19:57:01.615717 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.615658    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 19:57:01.615717 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.615668    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientPID"
Sep 18 19:57:01.616223 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.616208    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 19:57:01.616223 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.616222    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 19:57:01.616341 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.616228    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 19:57:01.616341 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.616239    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 19:57:01.616922 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.616909    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 19:57:01.616968 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.616933    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 19:57:01.616968 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.616943    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientPID"
Sep 18 19:57:01.618352 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.618339    2567 policy_none.go:49] "None policy: Start"
Sep 18 19:57:01.618419 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.618357    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 19:57:01.618419 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.618371    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 19:57:01.620332 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.620318    2567 policy_none.go:47] "Start"
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.661309    2567 manager.go:337] "Starting Device Plugin manager"
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.661484    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.661498    2567 server.go:87] "Starting device plugin registration server"
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.661798    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.661817    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.661899    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.661980    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.661986    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.662555    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 19:57:01.672515 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.662607    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:01.691203 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.691168    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 19:57:01.692288 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.692261    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 19:57:01.692389 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.692312    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 19:57:01.692389 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.692334    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 19:57:01.692389 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.692370    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 19:57:01.694735 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.694713    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 19:57:01.762971 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.762931    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 19:57:01.763972 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.763957    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 19:57:01.764037 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.763987    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 19:57:01.764037 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.764001    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientPID"
Sep 18 19:57:01.764037 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.764030    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.776273 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.776230    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.776273 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.776256    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-248.ec2.internal\": node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:01.792483 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.792457    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"]
Sep 18 19:57:01.792665 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.792526    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 19:57:01.794128 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.794112    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 19:57:01.794198 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.794142    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 19:57:01.794198 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.794151    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientPID"
Sep 18 19:57:01.796440 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.796427    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 19:57:01.796578 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.796564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.796619 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.796603    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 19:57:01.797386 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.797360    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 19:57:01.797386 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.797381    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 19:57:01.797386 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.797388    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 19:57:01.797589 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.797399    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientPID"
Sep 18 19:57:01.797589 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.797402    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 19:57:01.797589 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.797415    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientPID"
Sep 18 19:57:01.799091 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.799077    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:01.799705 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.799676    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.799797 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.799720    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 19:57:01.800524 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.800510    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 19:57:01.800600 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.800530    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 19:57:01.800600 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.800544    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeHasSufficientPID"
Sep 18 19:57:01.819406 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.819384    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-248.ec2.internal\" not found" node="ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.823811 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.823795    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-248.ec2.internal\" not found" node="ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.885912 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.885881    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5775ae8d5cff8302601fb6bb6349719-config\") pod \"kube-apiserver-proxy-ip-10-0-130-248.ec2.internal\" (UID: \"a5775ae8d5cff8302601fb6bb6349719\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.885912 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.885911    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a2ad7a23666b3b01eca68b125c23c134-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal\" (UID: \"a2ad7a23666b3b01eca68b125c23c134\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.886114 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.885929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2ad7a23666b3b01eca68b125c23c134-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal\" (UID: \"a2ad7a23666b3b01eca68b125c23c134\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.900170 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:01.900131    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:01.986324 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.986290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a2ad7a23666b3b01eca68b125c23c134-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal\" (UID: \"a2ad7a23666b3b01eca68b125c23c134\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.986489 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.986327    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2ad7a23666b3b01eca68b125c23c134-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal\" (UID: \"a2ad7a23666b3b01eca68b125c23c134\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.986489 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.986351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5775ae8d5cff8302601fb6bb6349719-config\") pod \"kube-apiserver-proxy-ip-10-0-130-248.ec2.internal\" (UID: \"a5775ae8d5cff8302601fb6bb6349719\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.986489 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.986399    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a5775ae8d5cff8302601fb6bb6349719-config\") pod \"kube-apiserver-proxy-ip-10-0-130-248.ec2.internal\" (UID: \"a5775ae8d5cff8302601fb6bb6349719\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.986489 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.986407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a2ad7a23666b3b01eca68b125c23c134-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal\" (UID: \"a2ad7a23666b3b01eca68b125c23c134\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:01.986489 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:01.986407    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2ad7a23666b3b01eca68b125c23c134-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal\" (UID: \"a2ad7a23666b3b01eca68b125c23c134\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:02.000441 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.000410    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:02.101300 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.101230    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:02.123440 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.123414    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:02.127019 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.127001    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:02.201696 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.201639    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:02.302152 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.302121    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:02.402795 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.402712    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-248.ec2.internal\" not found"
Sep 18 19:57:02.425641 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.425615    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 19:57:02.484901 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.484860    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:02.499453 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.499434    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 19:57:02.501550 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.501535    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal"
Sep 18 19:57:02.512317 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.512301    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 19:57:02.535471 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.535446    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 19:57:02.535898 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.535585    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 19:57:02.535898 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.535627    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 19:57:02.559725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.559696    2567 apiserver.go:52] "Watching apiserver"
Sep 18 19:57:02.575569 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.575535    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 19:57:02.576094 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.576063    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-kd48x","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc","openshift-cluster-node-tuning-operator/tuned-9wwx2","openshift-image-registry/node-ca-sfbf2","openshift-multus/multus-additional-cni-plugins-rlnnq","openshift-multus/network-metrics-daemon-xcq27","openshift-ovn-kubernetes/ovnkube-node-k4gmk","kube-system/global-pull-secret-syncer-88w6c","kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal","openshift-dns/node-resolver-5pgn9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal","openshift-multus/multus-5t6m7","openshift-network-diagnostics/network-check-target-8n4xm","openshift-network-operator/iptables-alerter-f6ht7"]
Sep 18 19:57:02.579014 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.578993    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.581558 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.581530    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:02.581655 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.581636    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:02.582436 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.582380    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 19:57:02.582525 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.582453    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 19:57:02.584035 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.584011    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 19:57:02.584987 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.584767    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 19:57:02.584987 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.584832    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xk7s5\""
Sep 18 19:57:02.585150 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.585009    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 19:57:02.585150 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.585003    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 19:57:02.585236 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.585149    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.585236 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.585181    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 19:57:02.587228 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.587209    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.589320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b58b30b1-7c4f-456f-afc6-3a1893960606-tmp\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.589397 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-kubelet\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.589397 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-modprobe-d\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.589397 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589386    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.589397 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589393    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-sys\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.589556 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589420    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxhgl\" (UniqueName: \"kubernetes.io/projected/b58b30b1-7c4f-456f-afc6-3a1893960606-kube-api-access-hxhgl\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.589556 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589466    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f44e615-bda4-4723-af2c-a398fe9def0f-env-overrides\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.589556 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589517    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-kubernetes\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.589556 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589552    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-sysctl-conf\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.589766 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589574    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-var-lib-kubelet\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.589766 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589590    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-run-netns\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.589766 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589605    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-run-openvswitch\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.589766 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-run-ovn\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.589766 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589641    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-tuned\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.589766 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589663    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-systemd-units\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.589766 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk2qd\" (UniqueName: \"kubernetes.io/projected/3f44e615-bda4-4723-af2c-a398fe9def0f-kube-api-access-lk2qd\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.589766 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589751    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-sysctl-d\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589794    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-lib-modules\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-run-systemd\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589859    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-var-lib-openvswitch\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589893    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-etc-openvswitch\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589915    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589963    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-cni-bin\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.589987    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-cni-netd\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590011    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-log-socket\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f44e615-bda4-4723-af2c-a398fe9def0f-ovnkube-config\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590095 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f44e615-bda4-4723-af2c-a398fe9def0f-ovn-node-metrics-cert\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590491 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590112    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:02.590491 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590138    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-sysconfig\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.590491 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590170    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-systemd\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.590491 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590202    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-run\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.590491 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590237    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-slash\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590491 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590267    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590491 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590300    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f44e615-bda4-4723-af2c-a398fe9def0f-ovnkube-script-lib\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590491 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590333    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-node-log\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.590491 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.590355    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-host\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.592589 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.592030    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:02.592589 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.592116    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:02.592747 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.592628    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 19:57:02.592747 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.592671    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 19:57:02.593948 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.593927    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.594935 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.594913    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ml6zn\""
Sep 18 19:57:02.595061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.595040    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 19:57:02.595126 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.595066    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 19:57:02.595184 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.595140    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 19:57:02.595401 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.595388    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 19:57:02.595445 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.595425    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tdcqh\""
Sep 18 19:57:02.595758 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.595743    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 19:57:02.595872 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.595856    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 19:57:02.596467 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.596306    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:02.596910 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.596887    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 19:57:02.596996 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.596952    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6zxtn\""
Sep 18 19:57:02.598572 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.598557    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.600640 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.600620    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 19:57:02.600738 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.600646    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 19:57:02.600738 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.600710    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w29br\""
Sep 18 19:57:02.600842 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.600818    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 19:57:02.600942 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.600924    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 19:57:02.601010 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.600929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.601010 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.600991    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 19:57:02.601115 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.601019    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 19:57:02.601115 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.601060    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sb9rw\""
Sep 18 19:57:02.601252 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.601233    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 19:57:02.601318 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.601296    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 19:57:02.601370 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.601355    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gdtkw\""
Sep 18 19:57:02.602030 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.602017    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 19:57:02.603259 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.603246    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:02.603313 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.603295    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:02.604347 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.604330    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 19:57:02.604422 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.604409    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 19:57:02.604479 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.604443    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 19:57:02.604598 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.604586    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jb267\""
Sep 18 19:57:02.605568 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.605552    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.607778 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.607761    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lfcjv\""
Sep 18 19:57:02.607921 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.607907    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 19:57:02.612046 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.612022    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 19:52:01 +0000 UTC" deadline="2028-05-25 00:15:36.985063023 +0000 UTC"
Sep 18 19:57:02.612046 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.612044    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14740h18m34.373021325s"
Sep 18 19:57:02.637758 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.637734    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l7lq6"
Sep 18 19:57:02.648135 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.648113    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l7lq6"
Sep 18 19:57:02.686195 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.686168    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 19:57:02.690618 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690598    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/047630c1-c48c-4052-a80e-1933bbdc85e1-host-slash\") pod \"iptables-alerter-f6ht7\" (UID: \"047630c1-c48c-4052-a80e-1933bbdc85e1\") " pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.690742 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690628    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-system-cni-dir\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.690742 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690649    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-cnibin\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.690742 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690664    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9174a4c-1933-4c2c-a691-2a647c2ec445-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.690742 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690699    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-host\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.690742 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690717    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b58b30b1-7c4f-456f-afc6-3a1893960606-tmp\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.690923 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-kubelet\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.690923 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690811    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-host\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.690923 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-kubelet\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.690923 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690860    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-device-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.690923 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690890    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9174a4c-1933-4c2c-a691-2a647c2ec445-cni-binary-copy\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.690923 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690916    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-run-multus-certs\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-modprobe-d\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690967    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-sys\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.690991    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f44e615-bda4-4723-af2c-a398fe9def0f-env-overrides\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691011    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691017    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdz97\" (UniqueName: \"kubernetes.io/projected/6f6ea4ea-23bb-4743-aba1-573297986157-kube-api-access-mdz97\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691040    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9174a4c-1933-4c2c-a691-2a647c2ec445-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691054    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-sys\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691060    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4cqj\" (UniqueName: \"kubernetes.io/projected/d9174a4c-1933-4c2c-a691-2a647c2ec445-kube-api-access-b4cqj\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691080    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-modprobe-d\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a4a019a5-9f6b-4c63-bd03-a28117dea38e-agent-certs\") pod \"konnectivity-agent-kd48x\" (UID: \"a4a019a5-9f6b-4c63-bd03-a28117dea38e\") " pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:02.691164 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-var-lib-kubelet\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691181    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691206    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-etc-selinux\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691213    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-var-lib-kubelet\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691230    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691254    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691279    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-hostroot\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-run-netns\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-run-ovn\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/79818bbf-4ed9-4ffe-ace4-dd7492917365-hosts-file\") pod \"node-resolver-5pgn9\" (UID: \"79818bbf-4ed9-4ffe-ace4-dd7492917365\") " pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691393    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-run-netns\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691395    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-run-netns\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691400    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-run-ovn\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691415    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-lib-modules\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691439    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f44e615-bda4-4723-af2c-a398fe9def0f-env-overrides\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691451    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-run-systemd\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691481    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-etc-openvswitch\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.691681 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691493    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-run-systemd\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691510    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-lib-modules\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691533    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691531    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-etc-openvswitch\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-cni-netd\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691578    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9174a4c-1933-4c2c-a691-2a647c2ec445-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691607    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg5xr\" (UniqueName: \"kubernetes.io/projected/79818bbf-4ed9-4ffe-ace4-dd7492917365-kube-api-access-rg5xr\") pod \"node-resolver-5pgn9\" (UID: \"79818bbf-4ed9-4ffe-ace4-dd7492917365\") " pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691608    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-cni-netd\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-cni-dir\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691649    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f44e615-bda4-4723-af2c-a398fe9def0f-ovn-node-metrics-cert\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691667    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da2a6760-9996-4138-a34d-3c41b2c98d10-cni-binary-copy\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691701    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-sysconfig\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691730    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-run\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f44e615-bda4-4723-af2c-a398fe9def0f-ovnkube-script-lib\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691812    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-sysconfig\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691848    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-run\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.692369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691848    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/adcd525f-a8d8-4903-95d6-37a8b8397f60-host\") pod \"node-ca-sfbf2\" (UID: \"adcd525f-a8d8-4903-95d6-37a8b8397f60\") " pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691872    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/adcd525f-a8d8-4903-95d6-37a8b8397f60-serviceca\") pod \"node-ca-sfbf2\" (UID: \"adcd525f-a8d8-4903-95d6-37a8b8397f60\") " pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691887    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-dbus\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691903    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-var-lib-kubelet\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691925    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-node-log\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691959    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-node-log\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.691982    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzh8g\" (UniqueName: \"kubernetes.io/projected/adcd525f-a8d8-4903-95d6-37a8b8397f60-kube-api-access-rzh8g\") pod \"node-ca-sfbf2\" (UID: \"adcd525f-a8d8-4903-95d6-37a8b8397f60\") " pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692039    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrdwc\" (UniqueName: \"kubernetes.io/projected/047630c1-c48c-4052-a80e-1933bbdc85e1-kube-api-access-vrdwc\") pod \"iptables-alerter-f6ht7\" (UID: \"047630c1-c48c-4052-a80e-1933bbdc85e1\") " pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692060    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-etc-kubernetes\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692084    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxhgl\" (UniqueName: \"kubernetes.io/projected/b58b30b1-7c4f-456f-afc6-3a1893960606-kube-api-access-hxhgl\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692099    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9174a4c-1933-4c2c-a691-2a647c2ec445-cnibin\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692113    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-daemon-config\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692126    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w42sj\" (UniqueName: \"kubernetes.io/projected/da2a6760-9996-4138-a34d-3c41b2c98d10-kube-api-access-w42sj\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692147    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a4a019a5-9f6b-4c63-bd03-a28117dea38e-konnectivity-ca\") pod \"konnectivity-agent-kd48x\" (UID: \"a4a019a5-9f6b-4c63-bd03-a28117dea38e\") " pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-socket-dir-parent\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692219    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f44e615-bda4-4723-af2c-a398fe9def0f-ovnkube-script-lib\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692223    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-conf-dir\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.693061 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-kubernetes\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692305    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-sysctl-conf\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692332    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-var-lib-cni-multus\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692358    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-registration-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692383    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9174a4c-1933-4c2c-a691-2a647c2ec445-system-cni-dir\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692358    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-kubernetes\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692409    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-run-openvswitch\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692425    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-sysctl-conf\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692436    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-tuned\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692464    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-run-openvswitch\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692474    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-systemd-units\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692501    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk2qd\" (UniqueName: \"kubernetes.io/projected/3f44e615-bda4-4723-af2c-a398fe9def0f-kube-api-access-lk2qd\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-socket-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692547    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfk8z\" (UniqueName: \"kubernetes.io/projected/350e1329-41a8-4b7d-931f-70c1272e9e88-kube-api-access-lfk8z\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692574    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-sysctl-d\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692706    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-var-lib-openvswitch\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692733    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-systemd-units\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.693908 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692737    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-sysctl-d\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-cni-bin\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692810    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f44e615-bda4-4723-af2c-a398fe9def0f-ovnkube-config\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692826    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-cni-bin\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692856    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-sys-fs\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692945    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-var-lib-openvswitch\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.692977    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/047630c1-c48c-4052-a80e-1933bbdc85e1-iptables-alerter-script\") pod \"iptables-alerter-f6ht7\" (UID: \"047630c1-c48c-4052-a80e-1933bbdc85e1\") " pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693001    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79818bbf-4ed9-4ffe-ace4-dd7492917365-tmp-dir\") pod \"node-resolver-5pgn9\" (UID: \"79818bbf-4ed9-4ffe-ace4-dd7492917365\") " pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693030    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-kubelet-config\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693057    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-log-socket\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693076    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-run-k8s-cni-cncf-io\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-var-lib-cni-bin\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-systemd\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693185    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-systemd\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693226    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-log-socket\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693259    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-slash\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.694646 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693278    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f44e615-bda4-4723-af2c-a398fe9def0f-ovnkube-config\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.695137 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693295    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.695137 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-slash\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.695137 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.693398    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f44e615-bda4-4723-af2c-a398fe9def0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.695137 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.694280    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b58b30b1-7c4f-456f-afc6-3a1893960606-tmp\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.695137 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.694552    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f44e615-bda4-4723-af2c-a398fe9def0f-ovn-node-metrics-cert\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.695137 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.694612    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b58b30b1-7c4f-456f-afc6-3a1893960606-etc-tuned\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.701066 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.701050    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 19:57:02.701066 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.701070    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 19:57:02.701183 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.701079    2567 projected.go:196] Error preparing data for projected volume kube-api-access-29hl2 for pod openshift-network-diagnostics/network-check-target-8n4xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:02.701183 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.701129    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2 podName:6873a0e0-ad43-4f29-bbd3-6dfddc5666a9 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:03.201114616 +0000 UTC m=+3.761698052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-29hl2" (UniqueName: "kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2") pod "network-check-target-8n4xm" (UID: "6873a0e0-ad43-4f29-bbd3-6dfddc5666a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:02.702577 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.702554    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxhgl\" (UniqueName: \"kubernetes.io/projected/b58b30b1-7c4f-456f-afc6-3a1893960606-kube-api-access-hxhgl\") pod \"tuned-9wwx2\" (UID: \"b58b30b1-7c4f-456f-afc6-3a1893960606\") " pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.702793 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.702777    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk2qd\" (UniqueName: \"kubernetes.io/projected/3f44e615-bda4-4723-af2c-a398fe9def0f-kube-api-access-lk2qd\") pod \"ovnkube-node-k4gmk\" (UID: \"3f44e615-bda4-4723-af2c-a398fe9def0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.764009 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:02.763967    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2ad7a23666b3b01eca68b125c23c134.slice/crio-926a16c19b95786a2fd85af8c47c9ec7a54ac53ad4c65cf659ebd2aa34bd6443 WatchSource:0}: Error finding container 926a16c19b95786a2fd85af8c47c9ec7a54ac53ad4c65cf659ebd2aa34bd6443: Status 404 returned error can't find the container with id 926a16c19b95786a2fd85af8c47c9ec7a54ac53ad4c65cf659ebd2aa34bd6443
Sep 18 19:57:02.764266 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:02.764248    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5775ae8d5cff8302601fb6bb6349719.slice/crio-0d85ca661124dbd8e5bdd46b1ddc0cb5a1f991774879971a82db510e8f22e7c6 WatchSource:0}: Error finding container 0d85ca661124dbd8e5bdd46b1ddc0cb5a1f991774879971a82db510e8f22e7c6: Status 404 returned error can't find the container with id 0d85ca661124dbd8e5bdd46b1ddc0cb5a1f991774879971a82db510e8f22e7c6
Sep 18 19:57:02.771509 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.771493    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 19:57:02.794430 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794405    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-var-lib-cni-multus\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794453    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-registration-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.794586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794470    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9174a4c-1933-4c2c-a691-2a647c2ec445-system-cni-dir\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.794586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794486    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-socket-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.794586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794501    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfk8z\" (UniqueName: \"kubernetes.io/projected/350e1329-41a8-4b7d-931f-70c1272e9e88-kube-api-access-lfk8z\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:02.794586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794517    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-sys-fs\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.794586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794511    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-var-lib-cni-multus\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794534    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/047630c1-c48c-4052-a80e-1933bbdc85e1-iptables-alerter-script\") pod \"iptables-alerter-f6ht7\" (UID: \"047630c1-c48c-4052-a80e-1933bbdc85e1\") " pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.794586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794538    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9174a4c-1933-4c2c-a691-2a647c2ec445-system-cni-dir\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.794586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794566    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-registration-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79818bbf-4ed9-4ffe-ace4-dd7492917365-tmp-dir\") pod \"node-resolver-5pgn9\" (UID: \"79818bbf-4ed9-4ffe-ace4-dd7492917365\") " pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794603    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-sys-fs\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794634    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-socket-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794644    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-kubelet-config\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794730    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-kubelet-config\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-run-k8s-cni-cncf-io\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794765    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-var-lib-cni-bin\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794769    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-run-k8s-cni-cncf-io\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794811    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/047630c1-c48c-4052-a80e-1933bbdc85e1-host-slash\") pod \"iptables-alerter-f6ht7\" (UID: \"047630c1-c48c-4052-a80e-1933bbdc85e1\") " pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794837    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-system-cni-dir\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794863    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-cnibin\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794866    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79818bbf-4ed9-4ffe-ace4-dd7492917365-tmp-dir\") pod \"node-resolver-5pgn9\" (UID: \"79818bbf-4ed9-4ffe-ace4-dd7492917365\") " pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794866    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/047630c1-c48c-4052-a80e-1933bbdc85e1-host-slash\") pod \"iptables-alerter-f6ht7\" (UID: \"047630c1-c48c-4052-a80e-1933bbdc85e1\") " pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794814    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-var-lib-cni-bin\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794888    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9174a4c-1933-4c2c-a691-2a647c2ec445-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794913    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-system-cni-dir\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-cnibin\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.794930 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794937    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-device-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794955    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9174a4c-1933-4c2c-a691-2a647c2ec445-cni-binary-copy\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794969    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-run-multus-certs\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794983    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-device-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.794990    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdz97\" (UniqueName: \"kubernetes.io/projected/6f6ea4ea-23bb-4743-aba1-573297986157-kube-api-access-mdz97\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795011    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9174a4c-1933-4c2c-a691-2a647c2ec445-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795018    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-run-multus-certs\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9174a4c-1933-4c2c-a691-2a647c2ec445-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795070    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4cqj\" (UniqueName: \"kubernetes.io/projected/d9174a4c-1933-4c2c-a691-2a647c2ec445-kube-api-access-b4cqj\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a4a019a5-9f6b-4c63-bd03-a28117dea38e-agent-certs\") pod \"konnectivity-agent-kd48x\" (UID: \"a4a019a5-9f6b-4c63-bd03-a28117dea38e\") " pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795115    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/047630c1-c48c-4052-a80e-1933bbdc85e1-iptables-alerter-script\") pod \"iptables-alerter-f6ht7\" (UID: \"047630c1-c48c-4052-a80e-1933bbdc85e1\") " pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795209    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795250    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-etc-selinux\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795273    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795289    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-kubelet-dir\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795386    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.795409    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:02.795725 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795413    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-hostroot\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/79818bbf-4ed9-4ffe-ace4-dd7492917365-hosts-file\") pod \"node-resolver-5pgn9\" (UID: \"79818bbf-4ed9-4ffe-ace4-dd7492917365\") " pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.795490    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs podName:350e1329-41a8-4b7d-931f-70c1272e9e88 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:03.29546957 +0000 UTC m=+3.856053014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs") pod "network-metrics-daemon-xcq27" (UID: "350e1329-41a8-4b7d-931f-70c1272e9e88") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795484    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f6ea4ea-23bb-4743-aba1-573297986157-etc-selinux\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795507    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9174a4c-1933-4c2c-a691-2a647c2ec445-cni-binary-copy\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795551    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9174a4c-1933-4c2c-a691-2a647c2ec445-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-run-netns\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795554    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-hostroot\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.795580    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795612    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-run-netns\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795622    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/79818bbf-4ed9-4ffe-ace4-dd7492917365-hosts-file\") pod \"node-resolver-5pgn9\" (UID: \"79818bbf-4ed9-4ffe-ace4-dd7492917365\") " pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9174a4c-1933-4c2c-a691-2a647c2ec445-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795699    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg5xr\" (UniqueName: \"kubernetes.io/projected/79818bbf-4ed9-4ffe-ace4-dd7492917365-kube-api-access-rg5xr\") pod \"node-resolver-5pgn9\" (UID: \"79818bbf-4ed9-4ffe-ace4-dd7492917365\") " pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795729    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-cni-dir\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:02.795754    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret podName:2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c nodeName:}" failed. No retries permitted until 2026-09-18 19:57:03.295726989 +0000 UTC m=+3.856310441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret") pod "global-pull-secret-syncer-88w6c" (UID: "2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795789    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da2a6760-9996-4138-a34d-3c41b2c98d10-cni-binary-copy\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795819    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/adcd525f-a8d8-4903-95d6-37a8b8397f60-host\") pod \"node-ca-sfbf2\" (UID: \"adcd525f-a8d8-4903-95d6-37a8b8397f60\") " pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795843    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/adcd525f-a8d8-4903-95d6-37a8b8397f60-serviceca\") pod \"node-ca-sfbf2\" (UID: \"adcd525f-a8d8-4903-95d6-37a8b8397f60\") " pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-cni-dir\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795864    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-dbus\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795895    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/adcd525f-a8d8-4903-95d6-37a8b8397f60-host\") pod \"node-ca-sfbf2\" (UID: \"adcd525f-a8d8-4903-95d6-37a8b8397f60\") " pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795916    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-var-lib-kubelet\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.795995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzh8g\" (UniqueName: \"kubernetes.io/projected/adcd525f-a8d8-4903-95d6-37a8b8397f60-kube-api-access-rzh8g\") pod \"node-ca-sfbf2\" (UID: \"adcd525f-a8d8-4903-95d6-37a8b8397f60\") " pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796038    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrdwc\" (UniqueName: \"kubernetes.io/projected/047630c1-c48c-4052-a80e-1933bbdc85e1-kube-api-access-vrdwc\") pod \"iptables-alerter-f6ht7\" (UID: \"047630c1-c48c-4052-a80e-1933bbdc85e1\") " pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796063    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-etc-kubernetes\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796089    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9174a4c-1933-4c2c-a691-2a647c2ec445-cnibin\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796124    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-daemon-config\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796122    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9174a4c-1933-4c2c-a691-2a647c2ec445-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796336    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-etc-kubernetes\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796377    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/adcd525f-a8d8-4903-95d6-37a8b8397f60-serviceca\") pod \"node-ca-sfbf2\" (UID: \"adcd525f-a8d8-4903-95d6-37a8b8397f60\") " pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796399    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-host-var-lib-kubelet\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796405    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-dbus\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w42sj\" (UniqueName: \"kubernetes.io/projected/da2a6760-9996-4138-a34d-3c41b2c98d10-kube-api-access-w42sj\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796448    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a4a019a5-9f6b-4c63-bd03-a28117dea38e-konnectivity-ca\") pod \"konnectivity-agent-kd48x\" (UID: \"a4a019a5-9f6b-4c63-bd03-a28117dea38e\") " pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:02.797176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796456    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/da2a6760-9996-4138-a34d-3c41b2c98d10-cni-binary-copy\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797665 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796448    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9174a4c-1933-4c2c-a691-2a647c2ec445-cnibin\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.797665 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796633    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-socket-dir-parent\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797665 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796656    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-conf-dir\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797665 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796726    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-conf-dir\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797665 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796732    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-socket-dir-parent\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797665 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/da2a6760-9996-4138-a34d-3c41b2c98d10-multus-daemon-config\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.797665 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.796963    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a4a019a5-9f6b-4c63-bd03-a28117dea38e-konnectivity-ca\") pod \"konnectivity-agent-kd48x\" (UID: \"a4a019a5-9f6b-4c63-bd03-a28117dea38e\") " pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:02.797877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.797745    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a4a019a5-9f6b-4c63-bd03-a28117dea38e-agent-certs\") pod \"konnectivity-agent-kd48x\" (UID: \"a4a019a5-9f6b-4c63-bd03-a28117dea38e\") " pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:02.815092 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.815065    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfk8z\" (UniqueName: \"kubernetes.io/projected/350e1329-41a8-4b7d-931f-70c1272e9e88-kube-api-access-lfk8z\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:02.815186 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.815131    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg5xr\" (UniqueName: \"kubernetes.io/projected/79818bbf-4ed9-4ffe-ace4-dd7492917365-kube-api-access-rg5xr\") pod \"node-resolver-5pgn9\" (UID: \"79818bbf-4ed9-4ffe-ace4-dd7492917365\") " pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:02.815186 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.815152    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrdwc\" (UniqueName: \"kubernetes.io/projected/047630c1-c48c-4052-a80e-1933bbdc85e1-kube-api-access-vrdwc\") pod \"iptables-alerter-f6ht7\" (UID: \"047630c1-c48c-4052-a80e-1933bbdc85e1\") " pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.815536 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.815518    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4cqj\" (UniqueName: \"kubernetes.io/projected/d9174a4c-1933-4c2c-a691-2a647c2ec445-kube-api-access-b4cqj\") pod \"multus-additional-cni-plugins-rlnnq\" (UID: \"d9174a4c-1933-4c2c-a691-2a647c2ec445\") " pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.824050 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.824027    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdz97\" (UniqueName: \"kubernetes.io/projected/6f6ea4ea-23bb-4743-aba1-573297986157-kube-api-access-mdz97\") pod \"aws-ebs-csi-driver-node-24jjc\" (UID: \"6f6ea4ea-23bb-4743-aba1-573297986157\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:02.824050 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.824039    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w42sj\" (UniqueName: \"kubernetes.io/projected/da2a6760-9996-4138-a34d-3c41b2c98d10-kube-api-access-w42sj\") pod \"multus-5t6m7\" (UID: \"da2a6760-9996-4138-a34d-3c41b2c98d10\") " pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:02.824159 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.824042    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzh8g\" (UniqueName: \"kubernetes.io/projected/adcd525f-a8d8-4903-95d6-37a8b8397f60-kube-api-access-rzh8g\") pod \"node-ca-sfbf2\" (UID: \"adcd525f-a8d8-4903-95d6-37a8b8397f60\") " pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.911125 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.911066    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:02.920374 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:02.920351    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f44e615_bda4_4723_af2c_a398fe9def0f.slice/crio-85248405f14162a728a80da47d56921058d0acfa2574e7bfd792d27332ce03ea WatchSource:0}: Error finding container 85248405f14162a728a80da47d56921058d0acfa2574e7bfd792d27332ce03ea: Status 404 returned error can't find the container with id 85248405f14162a728a80da47d56921058d0acfa2574e7bfd792d27332ce03ea
Sep 18 19:57:02.926473 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.926453    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9wwx2"
Sep 18 19:57:02.934167 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:02.934149    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb58b30b1_7c4f_456f_afc6_3a1893960606.slice/crio-f5be25eaf3bceee520bd6271abbe40d9f642b8128b1df26aa238d8d28e0d4dca WatchSource:0}: Error finding container f5be25eaf3bceee520bd6271abbe40d9f642b8128b1df26aa238d8d28e0d4dca: Status 404 returned error can't find the container with id f5be25eaf3bceee520bd6271abbe40d9f642b8128b1df26aa238d8d28e0d4dca
Sep 18 19:57:02.941280 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.941266    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 19:57:02.947967 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.947948    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sfbf2"
Sep 18 19:57:02.952036 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.952020    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rlnnq"
Sep 18 19:57:02.955913 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:02.955896    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadcd525f_a8d8_4903_95d6_37a8b8397f60.slice/crio-0fec0b75f74f1089daee69ef77c71ed218a071dd1eb0e85eec5d8933c92c899e WatchSource:0}: Error finding container 0fec0b75f74f1089daee69ef77c71ed218a071dd1eb0e85eec5d8933c92c899e: Status 404 returned error can't find the container with id 0fec0b75f74f1089daee69ef77c71ed218a071dd1eb0e85eec5d8933c92c899e
Sep 18 19:57:02.960155 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:02.960139    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9174a4c_1933_4c2c_a691_2a647c2ec445.slice/crio-8433dd2d0c17ad4e538220fd043da159e69a2cdf18cc4d20a680c05ad7074a32 WatchSource:0}: Error finding container 8433dd2d0c17ad4e538220fd043da159e69a2cdf18cc4d20a680c05ad7074a32: Status 404 returned error can't find the container with id 8433dd2d0c17ad4e538220fd043da159e69a2cdf18cc4d20a680c05ad7074a32
Sep 18 19:57:02.967172 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.967155    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f6ht7"
Sep 18 19:57:02.975921 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:02.975905    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047630c1_c48c_4052_a80e_1933bbdc85e1.slice/crio-ee2c7f5c15de4d3f949ca051bf979448fdbef21601bac5d800a6caaef4810cf0 WatchSource:0}: Error finding container ee2c7f5c15de4d3f949ca051bf979448fdbef21601bac5d800a6caaef4810cf0: Status 404 returned error can't find the container with id ee2c7f5c15de4d3f949ca051bf979448fdbef21601bac5d800a6caaef4810cf0
Sep 18 19:57:02.985599 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.985580    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:02.993867 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:02.993848    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a019a5_9f6b_4c63_bd03_a28117dea38e.slice/crio-ddd67e3eedb587ecf52b37ac6c96b06480decd8edddb095ac14498121f60941e WatchSource:0}: Error finding container ddd67e3eedb587ecf52b37ac6c96b06480decd8edddb095ac14498121f60941e: Status 404 returned error can't find the container with id ddd67e3eedb587ecf52b37ac6c96b06480decd8edddb095ac14498121f60941e
Sep 18 19:57:02.998453 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:02.998438    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 19:57:03.000097 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.000084    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc"
Sep 18 19:57:03.008495 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:03.008478    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6ea4ea_23bb_4743_aba1_573297986157.slice/crio-36fe2bd77b371aec1b9f53f26a18f0f50a0a209dd3bdc054d095ae053b4f6baa WatchSource:0}: Error finding container 36fe2bd77b371aec1b9f53f26a18f0f50a0a209dd3bdc054d095ae053b4f6baa: Status 404 returned error can't find the container with id 36fe2bd77b371aec1b9f53f26a18f0f50a0a209dd3bdc054d095ae053b4f6baa
Sep 18 19:57:03.009608 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.009598    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5pgn9"
Sep 18 19:57:03.014352 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.014336    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5t6m7"
Sep 18 19:57:03.017669 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:03.017650    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79818bbf_4ed9_4ffe_ace4_dd7492917365.slice/crio-95630422dd7d27dcd66f6bc12b17b50a61deee99f22e064f526b229529bc5b67 WatchSource:0}: Error finding container 95630422dd7d27dcd66f6bc12b17b50a61deee99f22e064f526b229529bc5b67: Status 404 returned error can't find the container with id 95630422dd7d27dcd66f6bc12b17b50a61deee99f22e064f526b229529bc5b67
Sep 18 19:57:03.021875 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:03.021860    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda2a6760_9996_4138_a34d_3c41b2c98d10.slice/crio-411eb876f6ea6c468bccc385e9df1350ab6bc1d2e904dd3468a36ac6bb3ca207 WatchSource:0}: Error finding container 411eb876f6ea6c468bccc385e9df1350ab6bc1d2e904dd3468a36ac6bb3ca207: Status 404 returned error can't find the container with id 411eb876f6ea6c468bccc385e9df1350ab6bc1d2e904dd3468a36ac6bb3ca207
Sep 18 19:57:03.300645 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.300569    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:03.300645 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.300629    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:03.300868 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.300659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:03.300868 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:03.300804    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 19:57:03.300868 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:03.300818    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:03.300868 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:03.300827    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 19:57:03.300868 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:03.300842    2567 projected.go:196] Error preparing data for projected volume kube-api-access-29hl2 for pod openshift-network-diagnostics/network-check-target-8n4xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:03.301094 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:03.300878    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret podName:2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c nodeName:}" failed. No retries permitted until 2026-09-18 19:57:04.300860594 +0000 UTC m=+4.861444043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret") pod "global-pull-secret-syncer-88w6c" (UID: "2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:03.301094 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:03.300897    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2 podName:6873a0e0-ad43-4f29-bbd3-6dfddc5666a9 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:04.300887475 +0000 UTC m=+4.861470916 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-29hl2" (UniqueName: "kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2") pod "network-check-target-8n4xm" (UID: "6873a0e0-ad43-4f29-bbd3-6dfddc5666a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:03.301094 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:03.300960    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:03.301094 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:03.301003    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs podName:350e1329-41a8-4b7d-931f-70c1272e9e88 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:04.300984664 +0000 UTC m=+4.861568102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs") pod "network-metrics-daemon-xcq27" (UID: "350e1329-41a8-4b7d-931f-70c1272e9e88") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:03.650838 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.650755    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 19:52:02 +0000 UTC" deadline="2028-04-27 06:08:10.610692545 +0000 UTC"
Sep 18 19:57:03.650838 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.650795    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14074h11m6.959901584s"
Sep 18 19:57:03.696026 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.695997    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:03.696163 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:03.696137    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:03.709430 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.709311    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlnnq" event={"ID":"d9174a4c-1933-4c2c-a691-2a647c2ec445","Type":"ContainerStarted","Data":"8433dd2d0c17ad4e538220fd043da159e69a2cdf18cc4d20a680c05ad7074a32"}
Sep 18 19:57:03.731547 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.731513    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sfbf2" event={"ID":"adcd525f-a8d8-4903-95d6-37a8b8397f60","Type":"ContainerStarted","Data":"0fec0b75f74f1089daee69ef77c71ed218a071dd1eb0e85eec5d8933c92c899e"}
Sep 18 19:57:03.758886 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.758853    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" event={"ID":"3f44e615-bda4-4723-af2c-a398fe9def0f","Type":"ContainerStarted","Data":"85248405f14162a728a80da47d56921058d0acfa2574e7bfd792d27332ce03ea"}
Sep 18 19:57:03.785634 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.785586    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal" event={"ID":"a2ad7a23666b3b01eca68b125c23c134","Type":"ContainerStarted","Data":"926a16c19b95786a2fd85af8c47c9ec7a54ac53ad4c65cf659ebd2aa34bd6443"}
Sep 18 19:57:03.806607 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.806578    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5pgn9" event={"ID":"79818bbf-4ed9-4ffe-ace4-dd7492917365","Type":"ContainerStarted","Data":"95630422dd7d27dcd66f6bc12b17b50a61deee99f22e064f526b229529bc5b67"}
Sep 18 19:57:03.814856 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.814829    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9wwx2" event={"ID":"b58b30b1-7c4f-456f-afc6-3a1893960606","Type":"ContainerStarted","Data":"f5be25eaf3bceee520bd6271abbe40d9f642b8128b1df26aa238d8d28e0d4dca"}
Sep 18 19:57:03.832794 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.831229    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal" event={"ID":"a5775ae8d5cff8302601fb6bb6349719","Type":"ContainerStarted","Data":"0d85ca661124dbd8e5bdd46b1ddc0cb5a1f991774879971a82db510e8f22e7c6"}
Sep 18 19:57:03.848339 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.848310    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5t6m7" event={"ID":"da2a6760-9996-4138-a34d-3c41b2c98d10","Type":"ContainerStarted","Data":"411eb876f6ea6c468bccc385e9df1350ab6bc1d2e904dd3468a36ac6bb3ca207"}
Sep 18 19:57:03.872515 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.872286    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 19:57:03.884463 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.883084    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc" event={"ID":"6f6ea4ea-23bb-4743-aba1-573297986157","Type":"ContainerStarted","Data":"36fe2bd77b371aec1b9f53f26a18f0f50a0a209dd3bdc054d095ae053b4f6baa"}
Sep 18 19:57:03.897488 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.897456    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kd48x" event={"ID":"a4a019a5-9f6b-4c63-bd03-a28117dea38e","Type":"ContainerStarted","Data":"ddd67e3eedb587ecf52b37ac6c96b06480decd8edddb095ac14498121f60941e"}
Sep 18 19:57:03.911605 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:03.911493    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f6ht7" event={"ID":"047630c1-c48c-4052-a80e-1933bbdc85e1","Type":"ContainerStarted","Data":"ee2c7f5c15de4d3f949ca051bf979448fdbef21601bac5d800a6caaef4810cf0"}
Sep 18 19:57:04.308527 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:04.307775    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:04.308527 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:04.307834    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:04.308527 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:04.307870    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:04.308527 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.308013    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:04.308527 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.308073    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret podName:2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c nodeName:}" failed. No retries permitted until 2026-09-18 19:57:06.308054577 +0000 UTC m=+6.868638014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret") pod "global-pull-secret-syncer-88w6c" (UID: "2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:04.308527 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.308479    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 19:57:04.308527 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.308503    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 19:57:04.308527 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.308515    2567 projected.go:196] Error preparing data for projected volume kube-api-access-29hl2 for pod openshift-network-diagnostics/network-check-target-8n4xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:04.309054 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.308579    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2 podName:6873a0e0-ad43-4f29-bbd3-6dfddc5666a9 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:06.308564448 +0000 UTC m=+6.869147886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-29hl2" (UniqueName: "kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2") pod "network-check-target-8n4xm" (UID: "6873a0e0-ad43-4f29-bbd3-6dfddc5666a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:04.309054 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.308646    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:04.309054 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.308678    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs podName:350e1329-41a8-4b7d-931f-70c1272e9e88 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:06.308667264 +0000 UTC m=+6.869250714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs") pod "network-metrics-daemon-xcq27" (UID: "350e1329-41a8-4b7d-931f-70c1272e9e88") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:04.580019 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:04.579945    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 19:57:04.651297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:04.651256    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 19:52:02 +0000 UTC" deadline="2028-04-09 17:22:44.507904028 +0000 UTC"
Sep 18 19:57:04.651297 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:04.651295    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13653h25m39.856613705s"
Sep 18 19:57:04.692948 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:04.692818    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:04.698332 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.697795    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:04.698332 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:04.698103    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:04.698332 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:04.698281    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:05.694927 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:05.694897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:05.695467 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:05.695036    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:06.324119 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:06.324078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:06.324304 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:06.324150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:06.324304 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:06.324190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:06.324304 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.324281    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 19:57:06.324425 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.324311    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 19:57:06.324425 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.324325    2567 projected.go:196] Error preparing data for projected volume kube-api-access-29hl2 for pod openshift-network-diagnostics/network-check-target-8n4xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:06.324425 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.324330    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:06.324425 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.324286    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:06.324425 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.324387    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2 podName:6873a0e0-ad43-4f29-bbd3-6dfddc5666a9 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:10.324369193 +0000 UTC m=+10.884952629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-29hl2" (UniqueName: "kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2") pod "network-check-target-8n4xm" (UID: "6873a0e0-ad43-4f29-bbd3-6dfddc5666a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:06.324425 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.324413    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs podName:350e1329-41a8-4b7d-931f-70c1272e9e88 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:10.32439643 +0000 UTC m=+10.884979880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs") pod "network-metrics-daemon-xcq27" (UID: "350e1329-41a8-4b7d-931f-70c1272e9e88") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:06.324611 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.324470    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret podName:2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c nodeName:}" failed. No retries permitted until 2026-09-18 19:57:10.324422172 +0000 UTC m=+10.885005607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret") pod "global-pull-secret-syncer-88w6c" (UID: "2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:06.693430 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:06.693356    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:06.693577 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.693491    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:06.693639 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:06.693586    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:06.693778 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:06.693753    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:07.693898 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:07.693862    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:07.694315 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:07.694004    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:08.693132 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:08.693099    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:08.693323 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:08.693225    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:08.693620 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:08.693593    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:08.693797 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:08.693769    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:09.694181 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:09.693602    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:09.694181 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:09.693823    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:10.357792 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:10.357752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:10.357966 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:10.357821    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:10.357966 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:10.357851    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:10.357966 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.357941    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 19:57:10.358152 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.357970    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 19:57:10.358152 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.357986    2567 projected.go:196] Error preparing data for projected volume kube-api-access-29hl2 for pod openshift-network-diagnostics/network-check-target-8n4xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:10.358152 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.358027    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:10.358380 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.358350    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2 podName:6873a0e0-ad43-4f29-bbd3-6dfddc5666a9 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:18.358029049 +0000 UTC m=+18.918612505 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-29hl2" (UniqueName: "kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2") pod "network-check-target-8n4xm" (UID: "6873a0e0-ad43-4f29-bbd3-6dfddc5666a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:10.358492 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.357970    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:10.358492 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.358483    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret podName:2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c nodeName:}" failed. No retries permitted until 2026-09-18 19:57:18.358453546 +0000 UTC m=+18.919036993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret") pod "global-pull-secret-syncer-88w6c" (UID: "2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:10.358668 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.358653    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs podName:350e1329-41a8-4b7d-931f-70c1272e9e88 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:18.358630219 +0000 UTC m=+18.919213661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs") pod "network-metrics-daemon-xcq27" (UID: "350e1329-41a8-4b7d-931f-70c1272e9e88") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:10.693396 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:10.692857    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:10.693396 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.692996    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:10.693396 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:10.693259    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:10.693396 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:10.693351    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:11.694156 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:11.694123    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:11.694600 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:11.694256    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:12.693183 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:12.693150    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:12.693365 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:12.693153    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:12.693365 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:12.693286    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:12.693476 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:12.693390    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:13.693211 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:13.693182    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:13.693565 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:13.693289    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:14.692904 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:14.692865    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:14.693140 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:14.692912    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:14.693140 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:14.692998    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:14.693140 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:14.693116    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:15.693532 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:15.693495    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:15.694032 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:15.693649    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:16.693467 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:16.693433    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:16.693467 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:16.693465    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:16.693947 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:16.693571    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:16.693947 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:16.693707    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:17.692818 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:17.692784    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:17.692986 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:17.692920    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:18.416821 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:18.416777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:18.416840    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:18.416868    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.416954    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.416983    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.417037    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs podName:350e1329-41a8-4b7d-931f-70c1272e9e88 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:34.417023584 +0000 UTC m=+34.977607032 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs") pod "network-metrics-daemon-xcq27" (UID: "350e1329-41a8-4b7d-931f-70c1272e9e88") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.416987    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.417068    2567 projected.go:196] Error preparing data for projected volume kube-api-access-29hl2 for pod openshift-network-diagnostics/network-check-target-8n4xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.417116    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2 podName:6873a0e0-ad43-4f29-bbd3-6dfddc5666a9 nodeName:}" failed. No retries permitted until 2026-09-18 19:57:34.417104649 +0000 UTC m=+34.977688083 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-29hl2" (UniqueName: "kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2") pod "network-check-target-8n4xm" (UID: "6873a0e0-ad43-4f29-bbd3-6dfddc5666a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.416963    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:18.417313 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.417165    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret podName:2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c nodeName:}" failed. No retries permitted until 2026-09-18 19:57:34.417153372 +0000 UTC m=+34.977736809 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret") pod "global-pull-secret-syncer-88w6c" (UID: "2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:18.693466 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:18.693393    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:18.693612 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:18.693393    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:18.693612 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.693516    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:18.693728 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:18.693603    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:19.693409 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:19.693377    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:19.693885 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:19.693528    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:20.693473 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:20.693433    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:20.693911 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:20.693550    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:20.693911 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:20.693604    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:20.693911 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:20.693717    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:21.694916 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:21.694107    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:21.694916 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:21.694387    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:21.961500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:21.961099    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5t6m7" event={"ID":"da2a6760-9996-4138-a34d-3c41b2c98d10","Type":"ContainerStarted","Data":"5d463724c5c7abe4cab794543c5c51c8e9688af0a051e1147289c295be416d37"}
Sep 18 19:57:21.965876 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:21.964644    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" event={"ID":"3f44e615-bda4-4723-af2c-a398fe9def0f","Type":"ContainerStarted","Data":"be9a19db679e15aa2527577e98991faf5138bc5fdcb5721a31845cbb39dc403b"}
Sep 18 19:57:21.965876 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:21.964740    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" event={"ID":"3f44e615-bda4-4723-af2c-a398fe9def0f","Type":"ContainerStarted","Data":"7472024e492ed6977981c156a939fe5136d1931bacf9baba044163b0671c087c"}
Sep 18 19:57:21.965876 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:21.964793    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" event={"ID":"3f44e615-bda4-4723-af2c-a398fe9def0f","Type":"ContainerStarted","Data":"8c0a6ae4ad44e6149633130770d354fe566f5c8ee9936c648b1b3abbbbae9725"}
Sep 18 19:57:21.968733 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:21.968598    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9wwx2" event={"ID":"b58b30b1-7c4f-456f-afc6-3a1893960606","Type":"ContainerStarted","Data":"2c958fc7def27df36435e04cdaac95ae4650ac460625dd7e45ca3b25344052d6"}
Sep 18 19:57:21.970945 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:21.970917    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal" event={"ID":"a5775ae8d5cff8302601fb6bb6349719","Type":"ContainerStarted","Data":"98510ad6211f6c73b13d5abf1f4b869adea50eb337db8335329d6c8844ee2c93"}
Sep 18 19:57:21.983544 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:21.983506    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5t6m7" podStartSLOduration=2.355907346 podStartE2EDuration="20.983493994s" totalImagesPullingTime="18.627586648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:21.983479306 +0000 UTC m=+22.544062763" watchObservedRunningTime="2026-09-18 19:57:21.983493994 +0000 UTC m=+22.544077450"
Sep 18 19:57:21.998832 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:21.998531    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-248.ec2.internal" podStartSLOduration=19.998513511 podStartE2EDuration="19.998513511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:21.997634449 +0000 UTC m=+22.558217907" watchObservedRunningTime="2026-09-18 19:57:21.998513511 +0000 UTC m=+22.559096969"
Sep 18 19:57:22.018568 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.018519    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9wwx2" podStartSLOduration=2.440268076 podStartE2EDuration="21.018500751s" totalImagesPullingTime="18.578232675s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:22.017959731 +0000 UTC m=+22.578543189" watchObservedRunningTime="2026-09-18 19:57:22.018500751 +0000 UTC m=+22.579084213"
Sep 18 19:57:22.692617 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.692591    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:22.692793 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.692591    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:22.692793 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:22.692702    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:22.692889 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:22.692798    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:22.977251 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.977215    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc" event={"ID":"6f6ea4ea-23bb-4743-aba1-573297986157","Type":"ContainerStarted","Data":"6ae10668d425a01e811a4febca70f481831fdfa485aa0efd9e0030c7a5c85479"}
Sep 18 19:57:22.978722 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.978702    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kd48x" event={"ID":"a4a019a5-9f6b-4c63-bd03-a28117dea38e","Type":"ContainerStarted","Data":"2684650fd1de0829847d610c484d88b2cf9f8d6467b50b9870c46c15c5bc1eb4"}
Sep 18 19:57:22.980243 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.980220    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f6ht7" event={"ID":"047630c1-c48c-4052-a80e-1933bbdc85e1","Type":"ContainerStarted","Data":"5594dae17490fceef891f0c6badec8da3a2c3d4d4a7c4e904e65f5d23cf6b0bf"}
Sep 18 19:57:22.981756 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.981735    2567 generic.go:356] "Generic (PLEG): container finished" podID="d9174a4c-1933-4c2c-a691-2a647c2ec445" containerID="ac398c09f9cf6deef8f693c074fd55e02ff2c133591e7b69b5a515fec29439cd" exitCode=0
Sep 18 19:57:22.981849 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.981800    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlnnq" event={"ID":"d9174a4c-1933-4c2c-a691-2a647c2ec445","Type":"ContainerDied","Data":"ac398c09f9cf6deef8f693c074fd55e02ff2c133591e7b69b5a515fec29439cd"}
Sep 18 19:57:22.983409 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.983313    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sfbf2" event={"ID":"adcd525f-a8d8-4903-95d6-37a8b8397f60","Type":"ContainerStarted","Data":"b8d2fb455a01e2665f977912a769c79309a259ccd056efb6b697257b4a39cd65"}
Sep 18 19:57:22.986315 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.986295    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" event={"ID":"3f44e615-bda4-4723-af2c-a398fe9def0f","Type":"ContainerStarted","Data":"01adba7331c3294bda08fbf9a7c97f0323b8de98730960fff216510096e23f70"}
Sep 18 19:57:22.986408 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.986346    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" event={"ID":"3f44e615-bda4-4723-af2c-a398fe9def0f","Type":"ContainerStarted","Data":"cadf8ff948d5b822a8a22999642b80301428f4eac5ee1b705412fd28f3318065"}
Sep 18 19:57:22.986408 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.986390    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" event={"ID":"3f44e615-bda4-4723-af2c-a398fe9def0f","Type":"ContainerStarted","Data":"6a20ae0dd6e258ebdca4b10ce0c004ac5846553a2cbcfc59e4913591e8faeb1d"}
Sep 18 19:57:22.987606 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.987578    2567 generic.go:356] "Generic (PLEG): container finished" podID="a2ad7a23666b3b01eca68b125c23c134" containerID="ec3d332e7d31b21b7399aca09e1f2756a6fae77cfea551ac61ddf7b4f98a85e4" exitCode=0
Sep 18 19:57:22.987717 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.987648    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal" event={"ID":"a2ad7a23666b3b01eca68b125c23c134","Type":"ContainerDied","Data":"ec3d332e7d31b21b7399aca09e1f2756a6fae77cfea551ac61ddf7b4f98a85e4"}
Sep 18 19:57:22.989534 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:22.989508    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5pgn9" event={"ID":"79818bbf-4ed9-4ffe-ace4-dd7492917365","Type":"ContainerStarted","Data":"0bc4d12e3d2c03c20e8b45a0b55774be8a7dc32e546ea1220a25ddc57ca2526a"}
Sep 18 19:57:23.030166 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.030124    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kd48x" podStartSLOduration=3.5124701050000002 podStartE2EDuration="22.030109469s" totalImagesPullingTime="18.517639364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:23.004931721 +0000 UTC m=+23.565515178" watchObservedRunningTime="2026-09-18 19:57:23.030109469 +0000 UTC m=+23.590692927"
Sep 18 19:57:23.060508 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.060334    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5pgn9" podStartSLOduration=3.567106649 podStartE2EDuration="22.060319119s" totalImagesPullingTime="18.49321247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:23.060151657 +0000 UTC m=+23.620735113" watchObservedRunningTime="2026-09-18 19:57:23.060319119 +0000 UTC m=+23.620902575"
Sep 18 19:57:23.120529 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.120486    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sfbf2" podStartSLOduration=3.5651190230000003 podStartE2EDuration="22.120472781s" totalImagesPullingTime="18.555353758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:23.119550905 +0000 UTC m=+23.680134363" watchObservedRunningTime="2026-09-18 19:57:23.120472781 +0000 UTC m=+23.681056235"
Sep 18 19:57:23.149728 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.149668    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f6ht7" podStartSLOduration=3.614709608 podStartE2EDuration="22.149651588s" totalImagesPullingTime="18.53494198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:23.149384626 +0000 UTC m=+23.709968085" watchObservedRunningTime="2026-09-18 19:57:23.149651588 +0000 UTC m=+23.710235046"
Sep 18 19:57:23.240332 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.240311    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 19:57:23.681552 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.681393    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T19:57:23.24032799Z","UUID":"08f318d2-2050-4b67-b0a8-8af819c9e657","Handler":null,"Name":"","Endpoint":""}
Sep 18 19:57:23.683384 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.683360    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 19:57:23.683524 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.683402    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 19:57:23.693330 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.693300    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:23.693455 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:23.693431    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:23.995273 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.995183    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc" event={"ID":"6f6ea4ea-23bb-4743-aba1-573297986157","Type":"ContainerStarted","Data":"3aa4176d2bb4c5a3595e57198a200322718607af7419d33714064e1b7adc7abf"}
Sep 18 19:57:23.997539 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:23.997471    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal" event={"ID":"a2ad7a23666b3b01eca68b125c23c134","Type":"ContainerStarted","Data":"28593c205f43bf7cbfc5a97d6308e56456b8278e3a94240ebb25c8e82be41bfd"}
Sep 18 19:57:24.692526 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:24.692495    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:24.692643 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:24.692529    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:24.692643 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:24.692595    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:24.692763 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:24.692669    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:25.001375 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:25.001339    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc" event={"ID":"6f6ea4ea-23bb-4743-aba1-573297986157","Type":"ContainerStarted","Data":"a55e0eb80cd460ec3b8b87caa06860c9539521a544dbe4ad9cbb1e60d3e3e783"}
Sep 18 19:57:25.004620 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:25.004596    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" event={"ID":"3f44e615-bda4-4723-af2c-a398fe9def0f","Type":"ContainerStarted","Data":"ef8b642159589fb6dce208f3468b6a8f7989976a7aeb1bdd7f192b3f2517bf0c"}
Sep 18 19:57:25.027368 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:25.027318    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-24jjc" podStartSLOduration=3.101265624 podStartE2EDuration="24.027304855s" totalImagesPullingTime="20.926039231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:25.027274838 +0000 UTC m=+25.587858295" watchObservedRunningTime="2026-09-18 19:57:25.027304855 +0000 UTC m=+25.587888289"
Sep 18 19:57:25.027940 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:25.027910    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-248.ec2.internal" podStartSLOduration=23.027894248 podStartE2EDuration="23.027894248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:24.015785591 +0000 UTC m=+24.576369047" watchObservedRunningTime="2026-09-18 19:57:25.027894248 +0000 UTC m=+25.588477706"
Sep 18 19:57:25.693098 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:25.693063    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:25.693258 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:25.693183    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:26.693286 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:26.693262    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:26.693729 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:26.693268    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:26.693729 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:26.693375    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:26.693729 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:26.693467    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:27.296976 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:27.296737    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:27.297350 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:27.297328    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:27.692564 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:27.692541    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:27.692708 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:27.692644    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:28.012957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:28.012874    2567 generic.go:356] "Generic (PLEG): container finished" podID="d9174a4c-1933-4c2c-a691-2a647c2ec445" containerID="19ab12bd18e2f4747d5b9eac76bb54e7975cbf887d2275df66ee25f5d942019e" exitCode=0
Sep 18 19:57:28.013790 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:28.012949    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlnnq" event={"ID":"d9174a4c-1933-4c2c-a691-2a647c2ec445","Type":"ContainerDied","Data":"19ab12bd18e2f4747d5b9eac76bb54e7975cbf887d2275df66ee25f5d942019e"}
Sep 18 19:57:28.016830 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:28.016804    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" event={"ID":"3f44e615-bda4-4723-af2c-a398fe9def0f","Type":"ContainerStarted","Data":"113297afde189d2637d2c3fda6332f6276da6a75447fb6a7959703fe86e11f12"}
Sep 18 19:57:28.016944 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:28.016839    2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 18 19:57:28.086806 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:28.086768    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk" podStartSLOduration=8.459959564 podStartE2EDuration="27.086755624s" totalImagesPullingTime="18.62679606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:28.086424074 +0000 UTC m=+28.647007531" watchObservedRunningTime="2026-09-18 19:57:28.086755624 +0000 UTC m=+28.647339080"
Sep 18 19:57:28.137227 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:28.137201    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:28.137816 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:28.137801    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kd48x"
Sep 18 19:57:28.693604 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:28.693449    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:28.693777 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:28.693495    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:28.693777 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:28.693716    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:28.693895 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:28.693789    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:29.019413 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.019392    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:29.019709 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.019460    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:29.019709 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.019518    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:29.033886 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.033863    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:29.034004 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.033991    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:57:29.088730 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.088708    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-88w6c"]
Sep 18 19:57:29.088826 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.088797    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:29.088916 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:29.088878    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:29.094591 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.094570    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xcq27"]
Sep 18 19:57:29.094726 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.094662    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:29.094790 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:29.094770    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:29.104538 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.104517    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8n4xm"]
Sep 18 19:57:29.104636 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:29.104596    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:29.104780 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:29.104755    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:30.022867 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:30.022835    2567 generic.go:356] "Generic (PLEG): container finished" podID="d9174a4c-1933-4c2c-a691-2a647c2ec445" containerID="3879cd7f2e7fe68c012f21c4b74346638a8a8e585e7c49f98b37d63eb48d0d12" exitCode=0
Sep 18 19:57:30.023288 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:30.022912    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlnnq" event={"ID":"d9174a4c-1933-4c2c-a691-2a647c2ec445","Type":"ContainerDied","Data":"3879cd7f2e7fe68c012f21c4b74346638a8a8e585e7c49f98b37d63eb48d0d12"}
Sep 18 19:57:30.693212 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:30.693184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:30.693347 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:30.693184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:30.693347 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:30.693307    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:30.693460 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:30.693184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:30.693460 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:30.693347    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:30.693460 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:30.693450    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:32.029072 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:32.029043    2567 generic.go:356] "Generic (PLEG): container finished" podID="d9174a4c-1933-4c2c-a691-2a647c2ec445" containerID="4385f2f443615fe7b6c13cf2570bcfeb9229159a95fa87a8e67da25c0aa1b05e" exitCode=0
Sep 18 19:57:32.029406 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:32.029092    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlnnq" event={"ID":"d9174a4c-1933-4c2c-a691-2a647c2ec445","Type":"ContainerDied","Data":"4385f2f443615fe7b6c13cf2570bcfeb9229159a95fa87a8e67da25c0aa1b05e"}
Sep 18 19:57:32.693008 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:32.692978    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:32.693008 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:32.692999    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:32.693192 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:32.693066    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:32.693192 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:32.693120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:32.693259 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:32.693189    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:32.693298 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:32.693245    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:34.434263 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:34.434228    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:34.434279    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:34.434343    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.434373    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.434449    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs podName:350e1329-41a8-4b7d-931f-70c1272e9e88 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:06.434428528 +0000 UTC m=+66.995011983 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs") pod "network-metrics-daemon-xcq27" (UID: "350e1329-41a8-4b7d-931f-70c1272e9e88") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.434454    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.434465    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.434481    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.434491    2567 projected.go:196] Error preparing data for projected volume kube-api-access-29hl2 for pod openshift-network-diagnostics/network-check-target-8n4xm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.434512    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret podName:2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c nodeName:}" failed. No retries permitted until 2026-09-18 19:58:06.434493621 +0000 UTC m=+66.995077074 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret") pod "global-pull-secret-syncer-88w6c" (UID: "2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c") : object "kube-system"/"original-pull-secret" not registered
Sep 18 19:57:34.434820 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.434530    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2 podName:6873a0e0-ad43-4f29-bbd3-6dfddc5666a9 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:06.434520744 +0000 UTC m=+66.995104179 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-29hl2" (UniqueName: "kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2") pod "network-check-target-8n4xm" (UID: "6873a0e0-ad43-4f29-bbd3-6dfddc5666a9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 19:57:34.692846 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:34.692754    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:34.692846 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:34.692756    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:34.693048 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:34.692878    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:34.693048 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.692872    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-88w6c" podUID="2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c"
Sep 18 19:57:34.693048 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.692978    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8n4xm" podUID="6873a0e0-ad43-4f29-bbd3-6dfddc5666a9"
Sep 18 19:57:34.693190 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:34.693076    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcq27" podUID="350e1329-41a8-4b7d-931f-70c1272e9e88"
Sep 18 19:57:35.503169 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.503086    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-248.ec2.internal" event="NodeReady"
Sep 18 19:57:35.503591 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.503370    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 19:57:35.570821 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.570792    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wtp9b"]
Sep 18 19:57:35.575571 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.575548    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-k4kgb"]
Sep 18 19:57:35.575735 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.575668    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.578815 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.578790    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k4kgb"
Sep 18 19:57:35.579052 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.579019    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 19:57:35.579157 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.579054    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 19:57:35.579604 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.579587    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2wjd\""
Sep 18 19:57:35.581909 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.581892    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 19:57:35.582282 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.582264    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 19:57:35.582365 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.582286    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 19:57:35.582422 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.582393    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xx2wd\""
Sep 18 19:57:35.587077 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.587034    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wtp9b"]
Sep 18 19:57:35.587868 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.587848    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k4kgb"]
Sep 18 19:57:35.642700 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.642669    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.642827 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.642735    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-config-volume\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.642827 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.642759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgnhj\" (UniqueName: \"kubernetes.io/projected/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-kube-api-access-lgnhj\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.642827 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.642783    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r89qb\" (UniqueName: \"kubernetes.io/projected/4fb83fb2-b7ef-4134-8f0b-556d934f2359-kube-api-access-r89qb\") pod \"ingress-canary-k4kgb\" (UID: \"4fb83fb2-b7ef-4134-8f0b-556d934f2359\") " pod="openshift-ingress-canary/ingress-canary-k4kgb"
Sep 18 19:57:35.642827 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.642799    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-tmp-dir\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.642989 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.642863    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fb83fb2-b7ef-4134-8f0b-556d934f2359-cert\") pod \"ingress-canary-k4kgb\" (UID: \"4fb83fb2-b7ef-4134-8f0b-556d934f2359\") " pod="openshift-ingress-canary/ingress-canary-k4kgb"
Sep 18 19:57:35.743660 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.743627    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-config-volume\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.743839 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.743669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgnhj\" (UniqueName: \"kubernetes.io/projected/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-kube-api-access-lgnhj\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.743839 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.743704    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r89qb\" (UniqueName: \"kubernetes.io/projected/4fb83fb2-b7ef-4134-8f0b-556d934f2359-kube-api-access-r89qb\") pod \"ingress-canary-k4kgb\" (UID: \"4fb83fb2-b7ef-4134-8f0b-556d934f2359\") " pod="openshift-ingress-canary/ingress-canary-k4kgb"
Sep 18 19:57:35.743839 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.743826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-tmp-dir\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.744005 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.743908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fb83fb2-b7ef-4134-8f0b-556d934f2359-cert\") pod \"ingress-canary-k4kgb\" (UID: \"4fb83fb2-b7ef-4134-8f0b-556d934f2359\") " pod="openshift-ingress-canary/ingress-canary-k4kgb"
Sep 18 19:57:35.744005 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.743952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.744185 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:35.744162    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 19:57:35.744256 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:35.744223    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls podName:076b49bd-a8a9-4595-8a08-fedc2b71d2ea nodeName:}" failed. No retries permitted until 2026-09-18 19:57:36.244205871 +0000 UTC m=+36.804789329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls") pod "dns-default-wtp9b" (UID: "076b49bd-a8a9-4595-8a08-fedc2b71d2ea") : secret "dns-default-metrics-tls" not found
Sep 18 19:57:35.744256 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.744167    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-tmp-dir\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.744362 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.744276    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-config-volume\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.748397 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.748272    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fb83fb2-b7ef-4134-8f0b-556d934f2359-cert\") pod \"ingress-canary-k4kgb\" (UID: \"4fb83fb2-b7ef-4134-8f0b-556d934f2359\") " pod="openshift-ingress-canary/ingress-canary-k4kgb"
Sep 18 19:57:35.757957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.757908    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgnhj\" (UniqueName: \"kubernetes.io/projected/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-kube-api-access-lgnhj\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:35.759698 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.759658    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r89qb\" (UniqueName: \"kubernetes.io/projected/4fb83fb2-b7ef-4134-8f0b-556d934f2359-kube-api-access-r89qb\") pod \"ingress-canary-k4kgb\" (UID: \"4fb83fb2-b7ef-4134-8f0b-556d934f2359\") " pod="openshift-ingress-canary/ingress-canary-k4kgb"
Sep 18 19:57:35.894357 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:35.894322    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-k4kgb"
Sep 18 19:57:36.248513 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.248483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:36.248642 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:36.248624    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 19:57:36.248715 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:36.248704    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls podName:076b49bd-a8a9-4595-8a08-fedc2b71d2ea nodeName:}" failed. No retries permitted until 2026-09-18 19:57:37.248671848 +0000 UTC m=+37.809255303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls") pod "dns-default-wtp9b" (UID: "076b49bd-a8a9-4595-8a08-fedc2b71d2ea") : secret "dns-default-metrics-tls" not found
Sep 18 19:57:36.305770 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.305718    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-k4kgb"]
Sep 18 19:57:36.308117 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:57:36.308093    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb83fb2_b7ef_4134_8f0b_556d934f2359.slice/crio-b79fad2ae178ba45b7e0e30b23b64d240dddccedd132eb60fc7191ed901be3e6 WatchSource:0}: Error finding container b79fad2ae178ba45b7e0e30b23b64d240dddccedd132eb60fc7191ed901be3e6: Status 404 returned error can't find the container with id b79fad2ae178ba45b7e0e30b23b64d240dddccedd132eb60fc7191ed901be3e6
Sep 18 19:57:36.692921 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.692895    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:57:36.693679 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.692897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:57:36.693679 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.692897    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:57:36.696565 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.696548    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gb8gb\""
Sep 18 19:57:36.696705 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.696551    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 19:57:36.696705 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.696555    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6rwzk\""
Sep 18 19:57:36.696705 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.696590    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 19:57:36.696705 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.696657    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 19:57:36.696956 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:36.696921    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 19:57:37.042731 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:37.042635    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k4kgb" event={"ID":"4fb83fb2-b7ef-4134-8f0b-556d934f2359","Type":"ContainerStarted","Data":"b79fad2ae178ba45b7e0e30b23b64d240dddccedd132eb60fc7191ed901be3e6"}
Sep 18 19:57:37.046223 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:37.046193    2567 generic.go:356] "Generic (PLEG): container finished" podID="d9174a4c-1933-4c2c-a691-2a647c2ec445" containerID="c32b260895cda1971ab52e0a1ec33225b99a7b5247f309d63e606cd503f409ac" exitCode=0
Sep 18 19:57:37.046355 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:37.046245    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlnnq" event={"ID":"d9174a4c-1933-4c2c-a691-2a647c2ec445","Type":"ContainerDied","Data":"c32b260895cda1971ab52e0a1ec33225b99a7b5247f309d63e606cd503f409ac"}
Sep 18 19:57:37.258102 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:37.258072    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:37.258264 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:37.258242    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 19:57:37.258339 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:37.258326    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls podName:076b49bd-a8a9-4595-8a08-fedc2b71d2ea nodeName:}" failed. No retries permitted until 2026-09-18 19:57:39.258305727 +0000 UTC m=+39.818889176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls") pod "dns-default-wtp9b" (UID: "076b49bd-a8a9-4595-8a08-fedc2b71d2ea") : secret "dns-default-metrics-tls" not found
Sep 18 19:57:38.052564 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:38.052333    2567 generic.go:356] "Generic (PLEG): container finished" podID="d9174a4c-1933-4c2c-a691-2a647c2ec445" containerID="f95f1075ce9dd9d850e3b7c778012053d07a52cb2b3a9a05a6c4ecfc55c7e872" exitCode=0
Sep 18 19:57:38.053055 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:38.052424    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlnnq" event={"ID":"d9174a4c-1933-4c2c-a691-2a647c2ec445","Type":"ContainerDied","Data":"f95f1075ce9dd9d850e3b7c778012053d07a52cb2b3a9a05a6c4ecfc55c7e872"}
Sep 18 19:57:39.055951 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:39.055879    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-k4kgb" event={"ID":"4fb83fb2-b7ef-4134-8f0b-556d934f2359","Type":"ContainerStarted","Data":"d166b86056389ed8527db74b1bc8b784d420255437afa0d660b38849197398ee"}
Sep 18 19:57:39.059001 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:39.058973    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rlnnq" event={"ID":"d9174a4c-1933-4c2c-a691-2a647c2ec445","Type":"ContainerStarted","Data":"bcaa115f7b2bd176ab95f1d0f12dc31e0400c3f43defee8b8d83690805ccc47b"}
Sep 18 19:57:39.073016 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:39.072963    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-k4kgb" podStartSLOduration=1.69938217 podStartE2EDuration="4.072948256s" totalImagesPullingTime="2.373566086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:39.072261558 +0000 UTC m=+39.632845037" watchObservedRunningTime="2026-09-18 19:57:39.072948256 +0000 UTC m=+39.633531714"
Sep 18 19:57:39.097713 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:39.097657    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rlnnq" podStartSLOduration=9.19444122 podStartE2EDuration="38.097639545s" totalImagesPullingTime="28.903198325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:57:39.095502164 +0000 UTC m=+39.656085621" watchObservedRunningTime="2026-09-18 19:57:39.097639545 +0000 UTC m=+39.658223001"
Sep 18 19:57:39.272794 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:39.272758    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:39.272951 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:39.272898    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 19:57:39.272990 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:39.272954    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls podName:076b49bd-a8a9-4595-8a08-fedc2b71d2ea nodeName:}" failed. No retries permitted until 2026-09-18 19:57:43.27293727 +0000 UTC m=+43.833520721 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls") pod "dns-default-wtp9b" (UID: "076b49bd-a8a9-4595-8a08-fedc2b71d2ea") : secret "dns-default-metrics-tls" not found
Sep 18 19:57:43.301505 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:43.301461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:43.301917 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:43.301614    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 19:57:43.301917 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:43.301678    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls podName:076b49bd-a8a9-4595-8a08-fedc2b71d2ea nodeName:}" failed. No retries permitted until 2026-09-18 19:57:51.301662419 +0000 UTC m=+51.862245854 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls") pod "dns-default-wtp9b" (UID: "076b49bd-a8a9-4595-8a08-fedc2b71d2ea") : secret "dns-default-metrics-tls" not found
Sep 18 19:57:51.351222 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:57:51.351176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:57:51.351648 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:51.351303    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 19:57:51.351648 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:57:51.351357    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls podName:076b49bd-a8a9-4595-8a08-fedc2b71d2ea nodeName:}" failed. No retries permitted until 2026-09-18 19:58:07.351343309 +0000 UTC m=+67.911926744 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls") pod "dns-default-wtp9b" (UID: "076b49bd-a8a9-4595-8a08-fedc2b71d2ea") : secret "dns-default-metrics-tls" not found
Sep 18 19:58:01.035227 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:01.035198    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k4gmk"
Sep 18 19:58:06.451042 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.451005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:58:06.451042 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.451052    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:58:06.451508 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.451075    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:58:06.453877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.453857    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 19:58:06.453948 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.453892    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 19:58:06.453994 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.453892    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 19:58:06.462359 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:06.462338    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 19:58:06.462421 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:06.462396    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs podName:350e1329-41a8-4b7d-931f-70c1272e9e88 nodeName:}" failed. No retries permitted until 2026-09-18 19:59:10.462379582 +0000 UTC m=+131.022963017 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs") pod "network-metrics-daemon-xcq27" (UID: "350e1329-41a8-4b7d-931f-70c1272e9e88") : secret "metrics-daemon-secret" not found
Sep 18 19:58:06.465194 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.465174    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 19:58:06.465299 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.465284    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c-original-pull-secret\") pod \"global-pull-secret-syncer-88w6c\" (UID: \"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c\") " pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:58:06.475492 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.475473    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29hl2\" (UniqueName: \"kubernetes.io/projected/6873a0e0-ad43-4f29-bbd3-6dfddc5666a9-kube-api-access-29hl2\") pod \"network-check-target-8n4xm\" (UID: \"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9\") " pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:58:06.716074 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.716013    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gb8gb\""
Sep 18 19:58:06.718680 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.718667    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-88w6c"
Sep 18 19:58:06.723811 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:06.723795    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:58:07.036734 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:07.036704    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-88w6c"]
Sep 18 19:58:07.039115 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:07.039085    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6d77f4_7f24_4b8e_b5ae_04e68a6c495c.slice/crio-b62795f8c857cf027339355778ed8a353117d3e2ee3a2a347b45a6cfe50e982d WatchSource:0}: Error finding container b62795f8c857cf027339355778ed8a353117d3e2ee3a2a347b45a6cfe50e982d: Status 404 returned error can't find the container with id b62795f8c857cf027339355778ed8a353117d3e2ee3a2a347b45a6cfe50e982d
Sep 18 19:58:07.054031 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:07.053999    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8n4xm"]
Sep 18 19:58:07.056789 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:07.056764    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6873a0e0_ad43_4f29_bbd3_6dfddc5666a9.slice/crio-63ada40eedb733b3474a76222f020fcccc980b60522690ce84a702a88b29a0a6 WatchSource:0}: Error finding container 63ada40eedb733b3474a76222f020fcccc980b60522690ce84a702a88b29a0a6: Status 404 returned error can't find the container with id 63ada40eedb733b3474a76222f020fcccc980b60522690ce84a702a88b29a0a6
Sep 18 19:58:07.123994 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:07.123969    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8n4xm" event={"ID":"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9","Type":"ContainerStarted","Data":"63ada40eedb733b3474a76222f020fcccc980b60522690ce84a702a88b29a0a6"}
Sep 18 19:58:07.125167 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:07.125142    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-88w6c" event={"ID":"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c","Type":"ContainerStarted","Data":"b62795f8c857cf027339355778ed8a353117d3e2ee3a2a347b45a6cfe50e982d"}
Sep 18 19:58:07.357773 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:07.357743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:58:07.357946 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:07.357857    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 19:58:07.357946 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:07.357915    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls podName:076b49bd-a8a9-4595-8a08-fedc2b71d2ea nodeName:}" failed. No retries permitted until 2026-09-18 19:58:39.357901127 +0000 UTC m=+99.918484575 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls") pod "dns-default-wtp9b" (UID: "076b49bd-a8a9-4595-8a08-fedc2b71d2ea") : secret "dns-default-metrics-tls" not found
Sep 18 19:58:12.140313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:12.140277    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8n4xm" event={"ID":"6873a0e0-ad43-4f29-bbd3-6dfddc5666a9","Type":"ContainerStarted","Data":"abe257fc705c77e066c6ae1ec1817510044459e7ade283c649f0813053a05610"}
Sep 18 19:58:12.140754 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:12.140409    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:58:12.141707 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:12.141669    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-88w6c" event={"ID":"2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c","Type":"ContainerStarted","Data":"85baef7db35c45f00f55a189421509503d583d4b770a7c93f36367551bba0c23"}
Sep 18 19:58:12.155270 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:12.155223    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8n4xm" podStartSLOduration=66.709885932 podStartE2EDuration="1m11.155207733s" totalImagesPullingTime="4.445321801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:12.154950087 +0000 UTC m=+72.715533544" watchObservedRunningTime="2026-09-18 19:58:12.155207733 +0000 UTC m=+72.715791191"
Sep 18 19:58:12.168747 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:12.168710    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-88w6c" podStartSLOduration=66.703039069 podStartE2EDuration="1m11.168698873s" totalImagesPullingTime="4.465659804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:12.168534053 +0000 UTC m=+72.729117511" watchObservedRunningTime="2026-09-18 19:58:12.168698873 +0000 UTC m=+72.729282321"
Sep 18 19:58:20.591075 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.591038    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"]
Sep 18 19:58:20.593978 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.593956    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.597760 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.597734    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 18 19:58:20.597998 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.597744    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 18 19:58:20.598054 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.598015    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 18 19:58:20.598571 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.598557    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jstlk\""
Sep 18 19:58:20.598849 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.598832    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 18 19:58:20.626867 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.626844    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"]
Sep 18 19:58:20.702331 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.702305    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"]
Sep 18 19:58:20.705311 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.705294    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-f7n5c"]
Sep 18 19:58:20.705422 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.705406    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.707931 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.707917    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.710127 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.709248    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 18 19:58:20.710127 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.709723    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 18 19:58:20.710127 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.709982    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 18 19:58:20.710127 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.710100    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-xwkhn\""
Sep 18 19:58:20.710768 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.710744    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lh6xk\""
Sep 18 19:58:20.711025 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.711007    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 18 19:58:20.711349 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.711332    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 18 19:58:20.711445 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.711416    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 18 19:58:20.711508 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.711419    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 18 19:58:20.712618 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.712292    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-688c76f68d-56dm8"]
Sep 18 19:58:20.712737 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.712657    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 18 19:58:20.715918 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.715895    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.720147 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.720129    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 18 19:58:20.722596 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.722580    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 19:58:20.723495 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.723480    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 19:58:20.723628 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.723613    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 19:58:20.723714 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.723613    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-466kg\""
Sep 18 19:58:20.727924 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.727904    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 19:58:20.729462 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.729424    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"]
Sep 18 19:58:20.737667 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.737620    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-f7n5c"]
Sep 18 19:58:20.738820 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.738770    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-688c76f68d-56dm8"]
Sep 18 19:58:20.746913 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.746892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c77ae59-6d5b-4d9e-a223-4394d7df9656-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-d7dls\" (UID: \"3c77ae59-6d5b-4d9e-a223-4394d7df9656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.747012 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.746927    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3c77ae59-6d5b-4d9e-a223-4394d7df9656-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-d7dls\" (UID: \"3c77ae59-6d5b-4d9e-a223-4394d7df9656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.747012 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.746946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf6l5\" (UniqueName: \"kubernetes.io/projected/3c77ae59-6d5b-4d9e-a223-4394d7df9656-kube-api-access-jf6l5\") pod \"kube-storage-version-migrator-operator-55889f7b47-d7dls\" (UID: \"3c77ae59-6d5b-4d9e-a223-4394d7df9656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.847590 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.847568    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3c77ae59-6d5b-4d9e-a223-4394d7df9656-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-d7dls\" (UID: \"3c77ae59-6d5b-4d9e-a223-4394d7df9656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.847704 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.847596    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf6l5\" (UniqueName: \"kubernetes.io/projected/3c77ae59-6d5b-4d9e-a223-4394d7df9656-kube-api-access-jf6l5\") pod \"kube-storage-version-migrator-operator-55889f7b47-d7dls\" (UID: \"3c77ae59-6d5b-4d9e-a223-4394d7df9656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.847704 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.847615    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8fx5\" (UniqueName: \"kubernetes.io/projected/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-kube-api-access-m8fx5\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.847704 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.847630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-ca-trust-extracted\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.847704 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.847652    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-trusted-ca\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.847913 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.847892    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s477b\" (UniqueName: \"kubernetes.io/projected/f5be3994-b3be-4530-bdde-d207770b129b-kube-api-access-s477b\") pod \"service-ca-operator-57cdc4b768-2npsm\" (UID: \"f5be3994-b3be-4530-bdde-d207770b129b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.847972 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.847943    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-trusted-ca\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.848024 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.847970    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-installation-pull-secrets\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.848024 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.847996    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-bound-sa-token\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.848102 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848023    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k74zs\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-kube-api-access-k74zs\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.848102 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848071    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-certificates\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.848175 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848105    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.848175 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848124    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-config\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.848175 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848152    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c77ae59-6d5b-4d9e-a223-4394d7df9656-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-d7dls\" (UID: \"3c77ae59-6d5b-4d9e-a223-4394d7df9656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.848175 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848167    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-serving-cert\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.848328 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848182    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f5be3994-b3be-4530-bdde-d207770b129b-serving-cert\") pod \"service-ca-operator-57cdc4b768-2npsm\" (UID: \"f5be3994-b3be-4530-bdde-d207770b129b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.848328 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848196    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f5be3994-b3be-4530-bdde-d207770b129b-config\") pod \"service-ca-operator-57cdc4b768-2npsm\" (UID: \"f5be3994-b3be-4530-bdde-d207770b129b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.848328 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848197    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3c77ae59-6d5b-4d9e-a223-4394d7df9656-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-d7dls\" (UID: \"3c77ae59-6d5b-4d9e-a223-4394d7df9656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.848328 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.848215    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-image-registry-private-configuration\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.850233 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.850213    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3c77ae59-6d5b-4d9e-a223-4394d7df9656-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-d7dls\" (UID: \"3c77ae59-6d5b-4d9e-a223-4394d7df9656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.856138 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.856121    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf6l5\" (UniqueName: \"kubernetes.io/projected/3c77ae59-6d5b-4d9e-a223-4394d7df9656-kube-api-access-jf6l5\") pod \"kube-storage-version-migrator-operator-55889f7b47-d7dls\" (UID: \"3c77ae59-6d5b-4d9e-a223-4394d7df9656\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.902191 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.902172    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949460    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-certificates\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949538    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949565    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-config\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949604    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-serving-cert\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949627    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f5be3994-b3be-4530-bdde-d207770b129b-serving-cert\") pod \"service-ca-operator-57cdc4b768-2npsm\" (UID: \"f5be3994-b3be-4530-bdde-d207770b129b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949650    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f5be3994-b3be-4530-bdde-d207770b129b-config\") pod \"service-ca-operator-57cdc4b768-2npsm\" (UID: \"f5be3994-b3be-4530-bdde-d207770b129b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949700    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-image-registry-private-configuration\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949749    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8fx5\" (UniqueName: \"kubernetes.io/projected/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-kube-api-access-m8fx5\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949770    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-ca-trust-extracted\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949796    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-trusted-ca\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949841    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s477b\" (UniqueName: \"kubernetes.io/projected/f5be3994-b3be-4530-bdde-d207770b129b-kube-api-access-s477b\") pod \"service-ca-operator-57cdc4b768-2npsm\" (UID: \"f5be3994-b3be-4530-bdde-d207770b129b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949878    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-trusted-ca\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949902    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-installation-pull-secrets\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949925    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-bound-sa-token\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.949948    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k74zs\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-kube-api-access-k74zs\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.950518 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.950151    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-certificates\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.951404 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:20.950989    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 19:58:20.951404 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:20.951006    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-688c76f68d-56dm8: secret "image-registry-tls" not found
Sep 18 19:58:20.951404 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:20.951059    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls podName:21c9fad7-16b9-4eac-a36e-9b6dbc2490fa nodeName:}" failed. No retries permitted until 2026-09-18 19:58:21.451040744 +0000 UTC m=+82.011624198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls") pod "image-registry-688c76f68d-56dm8" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa") : secret "image-registry-tls" not found
Sep 18 19:58:20.951585 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.951511    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-trusted-ca\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.951879 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.951853    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f5be3994-b3be-4530-bdde-d207770b129b-config\") pod \"service-ca-operator-57cdc4b768-2npsm\" (UID: \"f5be3994-b3be-4530-bdde-d207770b129b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.952881 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.952837    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-trusted-ca\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.954085 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.954058    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-ca-trust-extracted\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.954321 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.954297    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-config\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.954714 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.954675    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-image-registry-private-configuration\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.955013 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.954987    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-serving-cert\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.955587 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.955564    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f5be3994-b3be-4530-bdde-d207770b129b-serving-cert\") pod \"service-ca-operator-57cdc4b768-2npsm\" (UID: \"f5be3994-b3be-4530-bdde-d207770b129b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.956354 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.956322    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-installation-pull-secrets\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.969107 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.969066    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s477b\" (UniqueName: \"kubernetes.io/projected/f5be3994-b3be-4530-bdde-d207770b129b-kube-api-access-s477b\") pod \"service-ca-operator-57cdc4b768-2npsm\" (UID: \"f5be3994-b3be-4530-bdde-d207770b129b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:20.969217 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.969196    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8fx5\" (UniqueName: \"kubernetes.io/projected/6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da-kube-api-access-m8fx5\") pod \"console-operator-fd4bb46c-f7n5c\" (UID: \"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da\") " pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:20.969455 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.969439    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-bound-sa-token\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:20.969534 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:20.969518    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k74zs\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-kube-api-access-k74zs\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:21.019340 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:21.019321    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"
Sep 18 19:58:21.024404 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:21.024391    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:21.170165 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:21.170088    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls"]
Sep 18 19:58:21.172973 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:21.172948    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c77ae59_6d5b_4d9e_a223_4394d7df9656.slice/crio-4dab68bff5e271f298b8ae7928f04d631a143c21859cc795a1fb29fc42a0528d WatchSource:0}: Error finding container 4dab68bff5e271f298b8ae7928f04d631a143c21859cc795a1fb29fc42a0528d: Status 404 returned error can't find the container with id 4dab68bff5e271f298b8ae7928f04d631a143c21859cc795a1fb29fc42a0528d
Sep 18 19:58:21.291372 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:21.291348    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm"]
Sep 18 19:58:21.295942 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:21.295522    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-f7n5c"]
Sep 18 19:58:21.295942 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:21.295536    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5be3994_b3be_4530_bdde_d207770b129b.slice/crio-c100c7ecccdeff04017e42f09d87f651409441c8dd39f8a1faa4328a1da158c7 WatchSource:0}: Error finding container c100c7ecccdeff04017e42f09d87f651409441c8dd39f8a1faa4328a1da158c7: Status 404 returned error can't find the container with id c100c7ecccdeff04017e42f09d87f651409441c8dd39f8a1faa4328a1da158c7
Sep 18 19:58:21.453011 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:21.452948    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:21.453105 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:21.453059    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 19:58:21.453105 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:21.453070    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-688c76f68d-56dm8: secret "image-registry-tls" not found
Sep 18 19:58:21.453171 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:21.453113    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls podName:21c9fad7-16b9-4eac-a36e-9b6dbc2490fa nodeName:}" failed. No retries permitted until 2026-09-18 19:58:22.453100299 +0000 UTC m=+83.013683747 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls") pod "image-registry-688c76f68d-56dm8" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa") : secret "image-registry-tls" not found
Sep 18 19:58:22.168248 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:22.168211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls" event={"ID":"3c77ae59-6d5b-4d9e-a223-4394d7df9656","Type":"ContainerStarted","Data":"4dab68bff5e271f298b8ae7928f04d631a143c21859cc795a1fb29fc42a0528d"}
Sep 18 19:58:22.170133 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:22.170075    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c" event={"ID":"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da","Type":"ContainerStarted","Data":"3f6ed858a45b1de4593d969125a710b627f7d9d647d92477192d41e51eb7b035"}
Sep 18 19:58:22.171414 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:22.171391    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm" event={"ID":"f5be3994-b3be-4530-bdde-d207770b129b","Type":"ContainerStarted","Data":"c100c7ecccdeff04017e42f09d87f651409441c8dd39f8a1faa4328a1da158c7"}
Sep 18 19:58:22.462253 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:22.462169    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:22.462410 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:22.462310    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 19:58:22.462410 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:22.462326    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-688c76f68d-56dm8: secret "image-registry-tls" not found
Sep 18 19:58:22.462410 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:22.462396    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls podName:21c9fad7-16b9-4eac-a36e-9b6dbc2490fa nodeName:}" failed. No retries permitted until 2026-09-18 19:58:24.462375686 +0000 UTC m=+85.022959128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls") pod "image-registry-688c76f68d-56dm8" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa") : secret "image-registry-tls" not found
Sep 18 19:58:23.932028 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.931988    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f9c2r"]
Sep 18 19:58:23.935604 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.935585    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:23.938034 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.938016    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 19:58:23.938423 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.938401    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dnrhk\""
Sep 18 19:58:23.938854 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.938835    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 19:58:23.939111 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.939089    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 19:58:23.939219 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.939203    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 19:58:23.952285 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.952263    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f9c2r"]
Sep 18 19:58:23.974078 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.974050    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:23.974223 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.974094    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffmcv\" (UniqueName: \"kubernetes.io/projected/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-api-access-ffmcv\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:23.974223 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.974175    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:23.974223 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.974202    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ba68aef-6f7a-4620-8af1-092ce821ea07-data-volume\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:23.974375 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:23.974229    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ba68aef-6f7a-4620-8af1-092ce821ea07-crio-socket\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.074903 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.074868    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.074903 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.074913    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ba68aef-6f7a-4620-8af1-092ce821ea07-data-volume\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.075156 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.074942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ba68aef-6f7a-4620-8af1-092ce821ea07-crio-socket\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.075156 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.075018    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 19:58:24.075156 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.075033    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.075156 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.075066    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffmcv\" (UniqueName: \"kubernetes.io/projected/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-api-access-ffmcv\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.075156 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.075107    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:24.57508583 +0000 UTC m=+85.135669280 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : configmap "kube-rbac-proxy" not found
Sep 18 19:58:24.075156 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.075142    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ba68aef-6f7a-4620-8af1-092ce821ea07-crio-socket\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.075400 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.075186    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:24.075400 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.075242    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:24.575224992 +0000 UTC m=+85.135808436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:24.075400 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.075271    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ba68aef-6f7a-4620-8af1-092ce821ea07-data-volume\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.086395 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.086372    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffmcv\" (UniqueName: \"kubernetes.io/projected/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-api-access-ffmcv\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.478370 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.478317    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:24.478477 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.478463    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 19:58:24.478519 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.478478    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-688c76f68d-56dm8: secret "image-registry-tls" not found
Sep 18 19:58:24.478552 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.478529    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls podName:21c9fad7-16b9-4eac-a36e-9b6dbc2490fa nodeName:}" failed. No retries permitted until 2026-09-18 19:58:28.478515607 +0000 UTC m=+89.039099056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls") pod "image-registry-688c76f68d-56dm8" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa") : secret "image-registry-tls" not found
Sep 18 19:58:24.579178 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.579149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.579286 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:24.579207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:24.579325 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.579293    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 19:58:24.579325 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.579301    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:24.579387 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.579351    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:25.579337964 +0000 UTC m=+86.139921412 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:24.579387 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:24.579365    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:25.579359538 +0000 UTC m=+86.139942986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : configmap "kube-rbac-proxy" not found
Sep 18 19:58:25.184129 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.184095    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls" event={"ID":"3c77ae59-6d5b-4d9e-a223-4394d7df9656","Type":"ContainerStarted","Data":"a8341aa347b13a5d0337c5af16a7dab23a7dd3c73ff07738a308a91614843857"}
Sep 18 19:58:25.185737 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.185716    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/0.log"
Sep 18 19:58:25.185852 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.185750    2567 generic.go:356] "Generic (PLEG): container finished" podID="6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da" containerID="b404505c7e1f872e5d4fe3cdd453a07417fdaf73c24f0da67a45febf70b0e4d1" exitCode=255
Sep 18 19:58:25.185852 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.185811    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c" event={"ID":"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da","Type":"ContainerDied","Data":"b404505c7e1f872e5d4fe3cdd453a07417fdaf73c24f0da67a45febf70b0e4d1"}
Sep 18 19:58:25.186050 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.186032    2567 scope.go:117] "RemoveContainer" containerID="b404505c7e1f872e5d4fe3cdd453a07417fdaf73c24f0da67a45febf70b0e4d1"
Sep 18 19:58:25.187366 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.187338    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm" event={"ID":"f5be3994-b3be-4530-bdde-d207770b129b","Type":"ContainerStarted","Data":"f92e4385e315f7d38f167b43319525a1b19dfc7b1bdbc916a8326950e8a8d3a0"}
Sep 18 19:58:25.214115 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.214066    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls" podStartSLOduration=2.150870119 podStartE2EDuration="5.21404883s" totalImagesPullingTime="3.063178711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:25.213086454 +0000 UTC m=+85.773669912" watchObservedRunningTime="2026-09-18 19:58:25.21404883 +0000 UTC m=+85.774632288"
Sep 18 19:58:25.300055 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.299999    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm" podStartSLOduration=2.354889617 podStartE2EDuration="5.299983681s" totalImagesPullingTime="2.945094064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:25.238148523 +0000 UTC m=+85.798731982" watchObservedRunningTime="2026-09-18 19:58:25.299983681 +0000 UTC m=+85.860567202"
Sep 18 19:58:25.586071 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.585975    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:25.586227 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.586083    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:25.586227 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:25.586125    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:25.586227 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:25.586175    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 19:58:25.586227 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:25.586186    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:27.586171419 +0000 UTC m=+88.146754881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:25.586227 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:25.586217    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:27.586205744 +0000 UTC m=+88.146789178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : configmap "kube-rbac-proxy" not found
Sep 18 19:58:25.621253 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:25.621231    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5pgn9_79818bbf-4ed9-4ffe-ace4-dd7492917365/dns-node-resolver/0.log"
Sep 18 19:58:26.192322 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:26.192292    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 19:58:26.192728 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:26.192714    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/0.log"
Sep 18 19:58:26.192777 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:26.192749    2567 generic.go:356] "Generic (PLEG): container finished" podID="6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da" containerID="548729ed2bf72162ee6edc71036f170f81da2eb51a3bed0a06fe0d551c55f84d" exitCode=255
Sep 18 19:58:26.192830 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:26.192816    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c" event={"ID":"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da","Type":"ContainerDied","Data":"548729ed2bf72162ee6edc71036f170f81da2eb51a3bed0a06fe0d551c55f84d"}
Sep 18 19:58:26.192879 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:26.192869    2567 scope.go:117] "RemoveContainer" containerID="b404505c7e1f872e5d4fe3cdd453a07417fdaf73c24f0da67a45febf70b0e4d1"
Sep 18 19:58:26.193073 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:26.193045    2567 scope.go:117] "RemoveContainer" containerID="548729ed2bf72162ee6edc71036f170f81da2eb51a3bed0a06fe0d551c55f84d"
Sep 18 19:58:26.193274 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:26.193248    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-f7n5c_openshift-console-operator(6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da)\"" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c" podUID="6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da"
Sep 18 19:58:27.021448 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.021420    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sfbf2_adcd525f-a8d8-4903-95d6-37a8b8397f60/node-ca/0.log"
Sep 18 19:58:27.196995 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.196972    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 19:58:27.197386 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.197268    2567 scope.go:117] "RemoveContainer" containerID="548729ed2bf72162ee6edc71036f170f81da2eb51a3bed0a06fe0d551c55f84d"
Sep 18 19:58:27.197435 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:27.197405    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-f7n5c_openshift-console-operator(6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da)\"" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c" podUID="6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da"
Sep 18 19:58:27.606190 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.606157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:27.606351 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.606222    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:27.606351 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:27.606311    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:27.606425 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:27.606376    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:31.606359564 +0000 UTC m=+92.166943012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:27.606425 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:27.606311    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 19:58:27.606497 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:27.606438    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:31.606426991 +0000 UTC m=+92.167010444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : configmap "kube-rbac-proxy" not found
Sep 18 19:58:27.622507 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.622487    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k4kgb_4fb83fb2-b7ef-4134-8f0b-556d934f2359/serve-healthcheck-canary/0.log"
Sep 18 19:58:27.790802 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.790772    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-6f62c"]
Sep 18 19:58:27.794880 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.794862    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:27.797328 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.797305    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 18 19:58:27.797328 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.797318    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hxjjl\""
Sep 18 19:58:27.797467 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.797311    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 18 19:58:27.797768 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.797755    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 18 19:58:27.797975 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.797961    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 18 19:58:27.802506 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.802488    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-6f62c"]
Sep 18 19:58:27.807327 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.807309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da22ed61-a0cb-45e2-8020-9fb7638bb9ec-signing-cabundle\") pod \"service-ca-55f6fc7b86-6f62c\" (UID: \"da22ed61-a0cb-45e2-8020-9fb7638bb9ec\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:27.807399 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.807350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da22ed61-a0cb-45e2-8020-9fb7638bb9ec-signing-key\") pod \"service-ca-55f6fc7b86-6f62c\" (UID: \"da22ed61-a0cb-45e2-8020-9fb7638bb9ec\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:27.807399 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.807370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddrvx\" (UniqueName: \"kubernetes.io/projected/da22ed61-a0cb-45e2-8020-9fb7638bb9ec-kube-api-access-ddrvx\") pod \"service-ca-55f6fc7b86-6f62c\" (UID: \"da22ed61-a0cb-45e2-8020-9fb7638bb9ec\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:27.908023 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.907946    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddrvx\" (UniqueName: \"kubernetes.io/projected/da22ed61-a0cb-45e2-8020-9fb7638bb9ec-kube-api-access-ddrvx\") pod \"service-ca-55f6fc7b86-6f62c\" (UID: \"da22ed61-a0cb-45e2-8020-9fb7638bb9ec\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:27.908121 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.908072    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da22ed61-a0cb-45e2-8020-9fb7638bb9ec-signing-cabundle\") pod \"service-ca-55f6fc7b86-6f62c\" (UID: \"da22ed61-a0cb-45e2-8020-9fb7638bb9ec\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:27.908176 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.908126    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da22ed61-a0cb-45e2-8020-9fb7638bb9ec-signing-key\") pod \"service-ca-55f6fc7b86-6f62c\" (UID: \"da22ed61-a0cb-45e2-8020-9fb7638bb9ec\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:27.908657 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.908637    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/da22ed61-a0cb-45e2-8020-9fb7638bb9ec-signing-cabundle\") pod \"service-ca-55f6fc7b86-6f62c\" (UID: \"da22ed61-a0cb-45e2-8020-9fb7638bb9ec\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:27.910392 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.910371    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/da22ed61-a0cb-45e2-8020-9fb7638bb9ec-signing-key\") pod \"service-ca-55f6fc7b86-6f62c\" (UID: \"da22ed61-a0cb-45e2-8020-9fb7638bb9ec\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:27.916366 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:27.916342    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddrvx\" (UniqueName: \"kubernetes.io/projected/da22ed61-a0cb-45e2-8020-9fb7638bb9ec-kube-api-access-ddrvx\") pod \"service-ca-55f6fc7b86-6f62c\" (UID: \"da22ed61-a0cb-45e2-8020-9fb7638bb9ec\") " pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:28.103259 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:28.103225    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c"
Sep 18 19:58:28.378021 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:28.377993    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-6f62c"]
Sep 18 19:58:28.381358 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:28.381325    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda22ed61_a0cb_45e2_8020_9fb7638bb9ec.slice/crio-a307a57f7ddc1c9075eba967273008e6b0202784a9b2931917a3a68c54ac78c6 WatchSource:0}: Error finding container a307a57f7ddc1c9075eba967273008e6b0202784a9b2931917a3a68c54ac78c6: Status 404 returned error can't find the container with id a307a57f7ddc1c9075eba967273008e6b0202784a9b2931917a3a68c54ac78c6
Sep 18 19:58:28.513319 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:28.513289    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:28.513436 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:28.513406    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 19:58:28.513436 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:28.513417    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-688c76f68d-56dm8: secret "image-registry-tls" not found
Sep 18 19:58:28.513508 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:28.513465    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls podName:21c9fad7-16b9-4eac-a36e-9b6dbc2490fa nodeName:}" failed. No retries permitted until 2026-09-18 19:58:36.513449519 +0000 UTC m=+97.074032955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls") pod "image-registry-688c76f68d-56dm8" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa") : secret "image-registry-tls" not found
Sep 18 19:58:29.205523 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:29.205492    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c" event={"ID":"da22ed61-a0cb-45e2-8020-9fb7638bb9ec","Type":"ContainerStarted","Data":"c3e9fd30cef55a1087f53e6b9d66596405be7f57fb7f4358a43de4784af99b41"}
Sep 18 19:58:29.205713 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:29.205542    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c" event={"ID":"da22ed61-a0cb-45e2-8020-9fb7638bb9ec","Type":"ContainerStarted","Data":"a307a57f7ddc1c9075eba967273008e6b0202784a9b2931917a3a68c54ac78c6"}
Sep 18 19:58:29.224781 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:29.224735    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-6f62c" podStartSLOduration=2.224716855 podStartE2EDuration="2.224716855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:29.224044403 +0000 UTC m=+89.784627860" watchObservedRunningTime="2026-09-18 19:58:29.224716855 +0000 UTC m=+89.785300312"
Sep 18 19:58:31.025504 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:31.025467    2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:31.025947 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:31.025522    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:31.025947 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:31.025896    2567 scope.go:117] "RemoveContainer" containerID="548729ed2bf72162ee6edc71036f170f81da2eb51a3bed0a06fe0d551c55f84d"
Sep 18 19:58:31.026076 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:31.026055    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-f7n5c_openshift-console-operator(6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da)\"" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c" podUID="6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da"
Sep 18 19:58:31.634019 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:31.633985    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:31.634187 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:31.634069    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:31.634187 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:31.634108    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 19:58:31.634187 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:31.634169    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:39.634154238 +0000 UTC m=+100.194737678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : configmap "kube-rbac-proxy" not found
Sep 18 19:58:31.634187 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:31.634181    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:31.634355 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:31.634231    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:39.634215603 +0000 UTC m=+100.194799057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:36.570451 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:36.570417    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:36.572746 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:36.572714    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") pod \"image-registry-688c76f68d-56dm8\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") " pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:36.629779 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:36.629746    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:36.907408 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:36.907381    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-688c76f68d-56dm8"]
Sep 18 19:58:36.910148 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:36.910123    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21c9fad7_16b9_4eac_a36e_9b6dbc2490fa.slice/crio-750bc0d5430231a366ee6629ddf960b0b3ec47ad2ca31ce532e5881b0c004c8b WatchSource:0}: Error finding container 750bc0d5430231a366ee6629ddf960b0b3ec47ad2ca31ce532e5881b0c004c8b: Status 404 returned error can't find the container with id 750bc0d5430231a366ee6629ddf960b0b3ec47ad2ca31ce532e5881b0c004c8b
Sep 18 19:58:37.232847 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:37.232764    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-688c76f68d-56dm8" event={"ID":"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa","Type":"ContainerStarted","Data":"ba98be91c1d8c4a73a5f4332dd02a1866f7dc1b1bc0533a890420f60fb820d20"}
Sep 18 19:58:37.232847 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:37.232827    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-688c76f68d-56dm8" event={"ID":"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa","Type":"ContainerStarted","Data":"750bc0d5430231a366ee6629ddf960b0b3ec47ad2ca31ce532e5881b0c004c8b"}
Sep 18 19:58:37.233025 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:37.232881    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:58:37.254858 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:37.254815    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-688c76f68d-56dm8" podStartSLOduration=17.254800106 podStartE2EDuration="17.254800106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:58:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:37.254212964 +0000 UTC m=+97.814796420" watchObservedRunningTime="2026-09-18 19:58:37.254800106 +0000 UTC m=+97.815383563"
Sep 18 19:58:39.392241 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:39.392211    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:58:39.394587 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:39.394562    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/076b49bd-a8a9-4595-8a08-fedc2b71d2ea-metrics-tls\") pod \"dns-default-wtp9b\" (UID: \"076b49bd-a8a9-4595-8a08-fedc2b71d2ea\") " pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:58:39.490827 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:39.490805    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r2wjd\""
Sep 18 19:58:39.499731 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:39.499714    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:58:39.695704 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:39.694640    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:39.695704 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:58:39.695199    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls podName:5ba68aef-6f7a-4620-8af1-092ce821ea07 nodeName:}" failed. No retries permitted until 2026-09-18 19:58:55.694747086 +0000 UTC m=+116.255330532 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f9c2r" (UID: "5ba68aef-6f7a-4620-8af1-092ce821ea07") : secret "insights-runtime-extractor-tls" not found
Sep 18 19:58:39.698004 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:39.695943    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:39.698004 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:39.696041    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:39.698004 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:39.696642    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ba68aef-6f7a-4620-8af1-092ce821ea07-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:39.782978 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:39.782949    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wtp9b"]
Sep 18 19:58:39.786860 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:39.786833    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076b49bd_a8a9_4595_8a08_fedc2b71d2ea.slice/crio-7871483372e3ef3955d8770a4d22850d6cf8f61540d3b8836a21d4b9b05b1e8f WatchSource:0}: Error finding container 7871483372e3ef3955d8770a4d22850d6cf8f61540d3b8836a21d4b9b05b1e8f: Status 404 returned error can't find the container with id 7871483372e3ef3955d8770a4d22850d6cf8f61540d3b8836a21d4b9b05b1e8f
Sep 18 19:58:40.244897 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:40.244856    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wtp9b" event={"ID":"076b49bd-a8a9-4595-8a08-fedc2b71d2ea","Type":"ContainerStarted","Data":"7871483372e3ef3955d8770a4d22850d6cf8f61540d3b8836a21d4b9b05b1e8f"}
Sep 18 19:58:42.253942 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:42.253913    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wtp9b" event={"ID":"076b49bd-a8a9-4595-8a08-fedc2b71d2ea","Type":"ContainerStarted","Data":"d6c9f475482964b81af3a7800fdaf8ebfaa2a20484c1950054c68cf259627a01"}
Sep 18 19:58:42.254327 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:42.253963    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wtp9b" event={"ID":"076b49bd-a8a9-4595-8a08-fedc2b71d2ea","Type":"ContainerStarted","Data":"69ff01926edfce2c16beb1e1be67cdd4812235e5cb42f785ced6bf05ed3d6580"}
Sep 18 19:58:42.254327 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:42.254051    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:58:42.271864 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:42.271825    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wtp9b" podStartSLOduration=65.251427151 podStartE2EDuration="1m7.271813302s" totalImagesPullingTime="2.020386151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:42.271664225 +0000 UTC m=+102.832247683" watchObservedRunningTime="2026-09-18 19:58:42.271813302 +0000 UTC m=+102.832396781"
Sep 18 19:58:43.146178 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:43.146150    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8n4xm"
Sep 18 19:58:45.692953 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:45.692923    2567 scope.go:117] "RemoveContainer" containerID="548729ed2bf72162ee6edc71036f170f81da2eb51a3bed0a06fe0d551c55f84d"
Sep 18 19:58:46.268806 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:46.268781    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 19:58:46.268958 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:46.268828    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c" event={"ID":"6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da","Type":"ContainerStarted","Data":"0ecce08b8d5223ae569657a7fdfcf267250dfff77bbfaefa27b315b878e11b16"}
Sep 18 19:58:46.269124 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:46.269103    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:46.289189 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:46.289137    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c" podStartSLOduration=23.349020119 podStartE2EDuration="26.289119949s" totalImagesPullingTime="2.94009983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:46.288204573 +0000 UTC m=+106.848788031" watchObservedRunningTime="2026-09-18 19:58:46.289119949 +0000 UTC m=+106.849703410"
Sep 18 19:58:46.308473 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:46.308448    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-f7n5c"
Sep 18 19:58:52.107832 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.107797    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"]
Sep 18 19:58:52.109742 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.109721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"
Sep 18 19:58:52.117290 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.117268    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 18 19:58:52.117828 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.117813    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jwfc7\""
Sep 18 19:58:52.118124 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.118107    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 18 19:58:52.156592 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.156567    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"]
Sep 18 19:58:52.177962 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.177938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6ad647b-347c-4d77-8a59-4420270b4209-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-44k5c\" (UID: \"c6ad647b-347c-4d77-8a59-4420270b4209\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"
Sep 18 19:58:52.178050 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.177971    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c6ad647b-347c-4d77-8a59-4420270b4209-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-44k5c\" (UID: \"c6ad647b-347c-4d77-8a59-4420270b4209\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"
Sep 18 19:58:52.203292 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.203269    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-688c76f68d-56dm8"]
Sep 18 19:58:52.207419 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.207388    2567 patch_prober.go:28] interesting pod/image-registry-688c76f68d-56dm8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 19:58:52.207528 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.207444    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-688c76f68d-56dm8" podUID="21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 19:58:52.260352 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.260329    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wtp9b"
Sep 18 19:58:52.279273 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.279250    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6ad647b-347c-4d77-8a59-4420270b4209-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-44k5c\" (UID: \"c6ad647b-347c-4d77-8a59-4420270b4209\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"
Sep 18 19:58:52.279358 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.279279    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c6ad647b-347c-4d77-8a59-4420270b4209-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-44k5c\" (UID: \"c6ad647b-347c-4d77-8a59-4420270b4209\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"
Sep 18 19:58:52.279867 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.279850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c6ad647b-347c-4d77-8a59-4420270b4209-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-44k5c\" (UID: \"c6ad647b-347c-4d77-8a59-4420270b4209\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"
Sep 18 19:58:52.281528 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.281507    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c6ad647b-347c-4d77-8a59-4420270b4209-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-44k5c\" (UID: \"c6ad647b-347c-4d77-8a59-4420270b4209\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"
Sep 18 19:58:52.348096 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.348071    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6678b7486f-zmglw"]
Sep 18 19:58:52.349841 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.349828    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.417680 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.417627    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"
Sep 18 19:58:52.443078 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.443053    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6678b7486f-zmglw"]
Sep 18 19:58:52.480148 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.480124    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44092adf-564e-4dfa-b9d7-1a9dc81417a1-bound-sa-token\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.480268 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.480214    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44092adf-564e-4dfa-b9d7-1a9dc81417a1-registry-tls\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.480268 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.480256    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqjtj\" (UniqueName: \"kubernetes.io/projected/44092adf-564e-4dfa-b9d7-1a9dc81417a1-kube-api-access-sqjtj\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.480374 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.480299    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/44092adf-564e-4dfa-b9d7-1a9dc81417a1-image-registry-private-configuration\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.480429 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.480399    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/44092adf-564e-4dfa-b9d7-1a9dc81417a1-installation-pull-secrets\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.480480 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.480431    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/44092adf-564e-4dfa-b9d7-1a9dc81417a1-registry-certificates\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.480480 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.480464    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/44092adf-564e-4dfa-b9d7-1a9dc81417a1-ca-trust-extracted\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.480584 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.480564    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44092adf-564e-4dfa-b9d7-1a9dc81417a1-trusted-ca\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.581454 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.581422    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44092adf-564e-4dfa-b9d7-1a9dc81417a1-trusted-ca\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.581454 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.581455    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44092adf-564e-4dfa-b9d7-1a9dc81417a1-bound-sa-token\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.581632 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.581488    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44092adf-564e-4dfa-b9d7-1a9dc81417a1-registry-tls\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.581632 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.581507    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqjtj\" (UniqueName: \"kubernetes.io/projected/44092adf-564e-4dfa-b9d7-1a9dc81417a1-kube-api-access-sqjtj\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.581632 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.581528    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/44092adf-564e-4dfa-b9d7-1a9dc81417a1-image-registry-private-configuration\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.581632 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.581590    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/44092adf-564e-4dfa-b9d7-1a9dc81417a1-installation-pull-secrets\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.581632 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.581617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/44092adf-564e-4dfa-b9d7-1a9dc81417a1-registry-certificates\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.581919 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.581643    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/44092adf-564e-4dfa-b9d7-1a9dc81417a1-ca-trust-extracted\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.582137 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.582112    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/44092adf-564e-4dfa-b9d7-1a9dc81417a1-ca-trust-extracted\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.582494 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.582469    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/44092adf-564e-4dfa-b9d7-1a9dc81417a1-registry-certificates\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.582731 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.582707    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44092adf-564e-4dfa-b9d7-1a9dc81417a1-trusted-ca\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.584153 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.584130    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/44092adf-564e-4dfa-b9d7-1a9dc81417a1-image-registry-private-configuration\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.584239 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.584164    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/44092adf-564e-4dfa-b9d7-1a9dc81417a1-registry-tls\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.584239 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.584130    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/44092adf-564e-4dfa-b9d7-1a9dc81417a1-installation-pull-secrets\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.625170 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.625147    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/44092adf-564e-4dfa-b9d7-1a9dc81417a1-bound-sa-token\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.639832 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.639807    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqjtj\" (UniqueName: \"kubernetes.io/projected/44092adf-564e-4dfa-b9d7-1a9dc81417a1-kube-api-access-sqjtj\") pod \"image-registry-6678b7486f-zmglw\" (UID: \"44092adf-564e-4dfa-b9d7-1a9dc81417a1\") " pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.658368 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.658341    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:52.719957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.719931    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-44k5c"]
Sep 18 19:58:52.722367 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:52.722339    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ad647b_347c_4d77_8a59_4420270b4209.slice/crio-19b529d5164d24c5d6e12ad76e111d08c9d8a24c95f93a3a8456ee40497442b5 WatchSource:0}: Error finding container 19b529d5164d24c5d6e12ad76e111d08c9d8a24c95f93a3a8456ee40497442b5: Status 404 returned error can't find the container with id 19b529d5164d24c5d6e12ad76e111d08c9d8a24c95f93a3a8456ee40497442b5
Sep 18 19:58:52.977821 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:52.977740    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6678b7486f-zmglw"]
Sep 18 19:58:52.982425 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:52.982400    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44092adf_564e_4dfa_b9d7_1a9dc81417a1.slice/crio-c33e9951ad8fa359bc48a20669c6e6e5a6a26d13c573632dd7b2c9dc87f9e8ad WatchSource:0}: Error finding container c33e9951ad8fa359bc48a20669c6e6e5a6a26d13c573632dd7b2c9dc87f9e8ad: Status 404 returned error can't find the container with id c33e9951ad8fa359bc48a20669c6e6e5a6a26d13c573632dd7b2c9dc87f9e8ad
Sep 18 19:58:53.293589 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:53.292911    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6678b7486f-zmglw" event={"ID":"44092adf-564e-4dfa-b9d7-1a9dc81417a1","Type":"ContainerStarted","Data":"3b919dc9a0650215c3f8fbcbecf579c941c2a1b2ade758dd229d1e8f14909020"}
Sep 18 19:58:53.293589 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:53.292976    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6678b7486f-zmglw" event={"ID":"44092adf-564e-4dfa-b9d7-1a9dc81417a1","Type":"ContainerStarted","Data":"c33e9951ad8fa359bc48a20669c6e6e5a6a26d13c573632dd7b2c9dc87f9e8ad"}
Sep 18 19:58:53.293589 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:53.293187    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:58:53.295236 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:53.295203    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c" event={"ID":"c6ad647b-347c-4d77-8a59-4420270b4209","Type":"ContainerStarted","Data":"19b529d5164d24c5d6e12ad76e111d08c9d8a24c95f93a3a8456ee40497442b5"}
Sep 18 19:58:53.338362 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:53.338275    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6678b7486f-zmglw" podStartSLOduration=1.3382590109999999 podStartE2EDuration="1.338259011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:58:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:53.338029903 +0000 UTC m=+113.898613357" watchObservedRunningTime="2026-09-18 19:58:53.338259011 +0000 UTC m=+113.898842468"
Sep 18 19:58:54.299477 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:54.299444    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c" event={"ID":"c6ad647b-347c-4d77-8a59-4420270b4209","Type":"ContainerStarted","Data":"568d3cddba92db54197e3e2dc55767a8bc0e49c8d6c669f8b59cf405da52cb95"}
Sep 18 19:58:54.317816 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:54.317776    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-44k5c" podStartSLOduration=1.233081698 podStartE2EDuration="2.317760576s" totalImagesPullingTime="1.084678878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:58:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:54.3172003 +0000 UTC m=+114.877783757" watchObservedRunningTime="2026-09-18 19:58:54.317760576 +0000 UTC m=+114.878344033"
Sep 18 19:58:55.706591 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:55.706557    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:55.708771 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:55.708749    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ba68aef-6f7a-4620-8af1-092ce821ea07-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f9c2r\" (UID: \"5ba68aef-6f7a-4620-8af1-092ce821ea07\") " pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:55.747020 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:55.746994    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f9c2r"
Sep 18 19:58:56.044256 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:56.044182    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f9c2r"]
Sep 18 19:58:56.047370 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:58:56.047339    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba68aef_6f7a_4620_8af1_092ce821ea07.slice/crio-08e8e5b162232d295076454038a13e48f3cae66ea2bf961131d410cc365893b1 WatchSource:0}: Error finding container 08e8e5b162232d295076454038a13e48f3cae66ea2bf961131d410cc365893b1: Status 404 returned error can't find the container with id 08e8e5b162232d295076454038a13e48f3cae66ea2bf961131d410cc365893b1
Sep 18 19:58:56.307339 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:56.307277    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f9c2r" event={"ID":"5ba68aef-6f7a-4620-8af1-092ce821ea07","Type":"ContainerStarted","Data":"2c5cda579d524fb8b84b00eedef482ccbb389cf65850c25cc64a938b7ac02f2a"}
Sep 18 19:58:56.307339 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:56.307330    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f9c2r" event={"ID":"5ba68aef-6f7a-4620-8af1-092ce821ea07","Type":"ContainerStarted","Data":"08e8e5b162232d295076454038a13e48f3cae66ea2bf961131d410cc365893b1"}
Sep 18 19:58:57.313234 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:57.313199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f9c2r" event={"ID":"5ba68aef-6f7a-4620-8af1-092ce821ea07","Type":"ContainerStarted","Data":"23fb6b0ebff28717fd75df62a792f3a9e858291a1cc80fe11bfdf5049be2f2ae"}
Sep 18 19:58:58.318463 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:58.318435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f9c2r" event={"ID":"5ba68aef-6f7a-4620-8af1-092ce821ea07","Type":"ContainerStarted","Data":"f4b008158108edd694981a542ceb986d19481cce7d9052ca071572c2f6bca6e6"}
Sep 18 19:58:58.343020 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:58:58.342977    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-f9c2r" podStartSLOduration=33.323434806 podStartE2EDuration="35.342964226s" totalImagesPullingTime="2.01952942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 19:58:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:58:58.342930229 +0000 UTC m=+118.903513686" watchObservedRunningTime="2026-09-18 19:58:58.342964226 +0000 UTC m=+118.903547682"
Sep 18 19:59:02.207843 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:02.207814    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:59:06.474866 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.474834    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-694794746c-6s9q5"]
Sep 18 19:59:06.479266 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.479248    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.482190 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.482169    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 19:59:06.482287 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.482171    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 19:59:06.482809 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.482789    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 19:59:06.482917 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.482903    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 19:59:06.482917 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.482908    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 19:59:06.483024 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.483009    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4v52r\""
Sep 18 19:59:06.483079 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.483015    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 19:59:06.487133 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.487114    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 19:59:06.488382 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.488362    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 19:59:06.491457 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.491435    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-694794746c-6s9q5"]
Sep 18 19:59:06.588564 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.588542    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tpzg\" (UniqueName: \"kubernetes.io/projected/22b10eae-315c-4097-8f35-345c43b51b5d-kube-api-access-7tpzg\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.588660 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.588591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-oauth-config\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.588660 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.588620    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-console-config\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.588799 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.588666    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-serving-cert\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.588799 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.588715    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-service-ca\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.588799 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.588742    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-trusted-ca-bundle\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.588799 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.588766    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-oauth-serving-cert\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.689497 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.689471    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-oauth-config\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.689602 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.689506    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-console-config\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.689602 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.689553    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-serving-cert\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.689602 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.689579    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-service-ca\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.689799 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.689605    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-trusted-ca-bundle\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.689799 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.689722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-oauth-serving-cert\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.689799 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.689792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tpzg\" (UniqueName: \"kubernetes.io/projected/22b10eae-315c-4097-8f35-345c43b51b5d-kube-api-access-7tpzg\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.690441 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.690419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-console-config\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.690538 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.690441    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-service-ca\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.690538 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.690507    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-trusted-ca-bundle\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.690659 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.690557    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-oauth-serving-cert\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.691944 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.691927    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-oauth-config\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.692054 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.692038    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-serving-cert\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.701750 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.701732    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tpzg\" (UniqueName: \"kubernetes.io/projected/22b10eae-315c-4097-8f35-345c43b51b5d-kube-api-access-7tpzg\") pod \"console-694794746c-6s9q5\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") " pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:06.788142 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:06.788087    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:07.068120 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:07.068043    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-694794746c-6s9q5"]
Sep 18 19:59:07.071767 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:59:07.071742    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22b10eae_315c_4097_8f35_345c43b51b5d.slice/crio-2f3b3f0fa8b35cd2870a01748bf2069ce25155cd0de61dd40ed93522192e9efd WatchSource:0}: Error finding container 2f3b3f0fa8b35cd2870a01748bf2069ce25155cd0de61dd40ed93522192e9efd: Status 404 returned error can't find the container with id 2f3b3f0fa8b35cd2870a01748bf2069ce25155cd0de61dd40ed93522192e9efd
Sep 18 19:59:07.351005 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:07.350974    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694794746c-6s9q5" event={"ID":"22b10eae-315c-4097-8f35-345c43b51b5d","Type":"ContainerStarted","Data":"2f3b3f0fa8b35cd2870a01748bf2069ce25155cd0de61dd40ed93522192e9efd"}
Sep 18 19:59:10.520222 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:10.520184    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:59:10.522759 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:10.522731    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350e1329-41a8-4b7d-931f-70c1272e9e88-metrics-certs\") pod \"network-metrics-daemon-xcq27\" (UID: \"350e1329-41a8-4b7d-931f-70c1272e9e88\") " pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:59:10.607904 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:10.607876    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6rwzk\""
Sep 18 19:59:10.616208 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:10.616187    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcq27"
Sep 18 19:59:11.096404 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:11.096240    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xcq27"]
Sep 18 19:59:11.098759 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:59:11.098728    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350e1329_41a8_4b7d_931f_70c1272e9e88.slice/crio-5af40dba5e7bcd6aad63f994714df14cd7ee0679854e659530f8d4ce7ccf425e WatchSource:0}: Error finding container 5af40dba5e7bcd6aad63f994714df14cd7ee0679854e659530f8d4ce7ccf425e: Status 404 returned error can't find the container with id 5af40dba5e7bcd6aad63f994714df14cd7ee0679854e659530f8d4ce7ccf425e
Sep 18 19:59:11.366624 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:11.366543    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694794746c-6s9q5" event={"ID":"22b10eae-315c-4097-8f35-345c43b51b5d","Type":"ContainerStarted","Data":"92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb"}
Sep 18 19:59:11.367801 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:11.367775    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xcq27" event={"ID":"350e1329-41a8-4b7d-931f-70c1272e9e88","Type":"ContainerStarted","Data":"5af40dba5e7bcd6aad63f994714df14cd7ee0679854e659530f8d4ce7ccf425e"}
Sep 18 19:59:11.385905 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:11.385865    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-694794746c-6s9q5" podStartSLOduration=1.564318863 podStartE2EDuration="5.385851127s" totalImagesPullingTime="3.821532264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:59:11.38535552 +0000 UTC m=+131.945938992" watchObservedRunningTime="2026-09-18 19:59:11.385851127 +0000 UTC m=+131.946434583"
Sep 18 19:59:12.372860 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:12.372825    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xcq27" event={"ID":"350e1329-41a8-4b7d-931f-70c1272e9e88","Type":"ContainerStarted","Data":"994bb9bdcb9cfd64e10e213e69dd5fb375736f2db46ca49a601a8d1c1ef15fbb"}
Sep 18 19:59:12.373222 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:12.372912    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xcq27" event={"ID":"350e1329-41a8-4b7d-931f-70c1272e9e88","Type":"ContainerStarted","Data":"8d12b645e2d25dc8a6ca09dda5ef0f778ce7dd456bf433f4c0a6e5ca64acece4"}
Sep 18 19:59:12.393521 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:12.393474    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xcq27" podStartSLOduration=130.565316269 podStartE2EDuration="2m11.393457434s" totalImagesPullingTime="828.141165ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:59:12.393322699 +0000 UTC m=+132.953906156" watchObservedRunningTime="2026-09-18 19:59:12.393457434 +0000 UTC m=+132.954040911"
Sep 18 19:59:14.303673 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:14.303647    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6678b7486f-zmglw"
Sep 18 19:59:15.085194 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.085158    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-whc66"]
Sep 18 19:59:15.088897 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.088883    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.098563 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.098544    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 18 19:59:15.099529 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.099510    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 19:59:15.099804 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.099792    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 19:59:15.099927 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.099909    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 18 19:59:15.099964 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.099933    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 18 19:59:15.099964 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.099944    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 19:59:15.100054 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.099909    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pp8xx\""
Sep 18 19:59:15.103973 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.103953    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9c582"]
Sep 18 19:59:15.107337 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.107320    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.112503 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.112481    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 19:59:15.112669 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.112642    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c6mwt\""
Sep 18 19:59:15.112795 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.112589    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 19:59:15.112795 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.112534    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 19:59:15.114215 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.114196    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-whc66"]
Sep 18 19:59:15.256539 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256508    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-textfile\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.256754 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.256754 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256566    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.256754 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256585    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.256754 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.256754 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256744    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a56f260-3fc2-4202-ae5c-86461eb42d71-sys\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.256957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256768    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-wtmp\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.256957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256807    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.256957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256831    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vltc\" (UniqueName: \"kubernetes.io/projected/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-api-access-4vltc\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.256957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-accelerators-collector-config\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.256957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256900    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a56f260-3fc2-4202-ae5c-86461eb42d71-root\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.256957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.256957 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256948    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a56f260-3fc2-4202-ae5c-86461eb42d71-metrics-client-ca\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.257198 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.256974    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsnbx\" (UniqueName: \"kubernetes.io/projected/0a56f260-3fc2-4202-ae5c-86461eb42d71-kube-api-access-wsnbx\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.257198 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.257031    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-tls\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357371 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357340    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.357371 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357374    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vltc\" (UniqueName: \"kubernetes.io/projected/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-api-access-4vltc\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357401    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-accelerators-collector-config\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a56f260-3fc2-4202-ae5c-86461eb42d71-root\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357479    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a56f260-3fc2-4202-ae5c-86461eb42d71-metrics-client-ca\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357526    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsnbx\" (UniqueName: \"kubernetes.io/projected/0a56f260-3fc2-4202-ae5c-86461eb42d71-kube-api-access-wsnbx\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357555    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-tls\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357582    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0a56f260-3fc2-4202-ae5c-86461eb42d71-root\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357596    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-textfile\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357724    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357760    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357791    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a56f260-3fc2-4202-ae5c-86461eb42d71-sys\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357839    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-wtmp\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.357907 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357864    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.358628 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357947    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-textfile\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.358628 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.357989    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-wtmp\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.358628 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.358038    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0a56f260-3fc2-4202-ae5c-86461eb42d71-sys\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.358628 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.358087    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-accelerators-collector-config\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.358628 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.358136    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0a56f260-3fc2-4202-ae5c-86461eb42d71-metrics-client-ca\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.358628 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.358215    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.358628 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.358568    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.360327 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.360290    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.360426 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.360404    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.364739 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.360761    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0a56f260-3fc2-4202-ae5c-86461eb42d71-node-exporter-tls\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.364739 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.360909    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.373792 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.373765    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsnbx\" (UniqueName: \"kubernetes.io/projected/0a56f260-3fc2-4202-ae5c-86461eb42d71-kube-api-access-wsnbx\") pod \"node-exporter-9c582\" (UID: \"0a56f260-3fc2-4202-ae5c-86461eb42d71\") " pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.374341 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.374323    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vltc\" (UniqueName: \"kubernetes.io/projected/8a18ab65-fbb1-408f-b7f4-9168e5ee4847-kube-api-access-4vltc\") pod \"kube-state-metrics-84c885658c-whc66\" (UID: \"8a18ab65-fbb1-408f-b7f4-9168e5ee4847\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.397586 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.397569    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66"
Sep 18 19:59:15.416248 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.416229    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9c582"
Sep 18 19:59:15.426332 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:59:15.426311    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a56f260_3fc2_4202_ae5c_86461eb42d71.slice/crio-ecab7d432ee49846918e8481e0e9946430b3135df0683e8a6aeceeaa28a6b9e0 WatchSource:0}: Error finding container ecab7d432ee49846918e8481e0e9946430b3135df0683e8a6aeceeaa28a6b9e0: Status 404 returned error can't find the container with id ecab7d432ee49846918e8481e0e9946430b3135df0683e8a6aeceeaa28a6b9e0
Sep 18 19:59:15.691234 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.691145    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-whc66"]
Sep 18 19:59:15.694373 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:59:15.694336    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a18ab65_fbb1_408f_b7f4_9168e5ee4847.slice/crio-c5014ff82c762e2cd8bd9c463079fab1e187ff9937c9bc08df3b2a84aa37df8f WatchSource:0}: Error finding container c5014ff82c762e2cd8bd9c463079fab1e187ff9937c9bc08df3b2a84aa37df8f: Status 404 returned error can't find the container with id c5014ff82c762e2cd8bd9c463079fab1e187ff9937c9bc08df3b2a84aa37df8f
Sep 18 19:59:15.826302 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.826275    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 19:59:15.831484 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.831467    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.835264 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.835243    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 19:59:15.835372 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.835356    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 19:59:15.835544 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.835532    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tp2nk\""
Sep 18 19:59:15.835609 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.835560    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 19:59:15.836090 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.836074    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 19:59:15.836280 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.836259    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 19:59:15.836386 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.836317    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 19:59:15.836445 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.836412    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 19:59:15.836991 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.836973    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 19:59:15.839972 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.839951    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 19:59:15.851311 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.851292    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 19:59:15.964148 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964076    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964148 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964116    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-web-config\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964419 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964155    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964419 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964192    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-out\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964419 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964217    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964419 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964239    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964419 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964358    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964419 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964411    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p4fn\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-kube-api-access-8p4fn\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964726 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964480    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964726 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964508    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964726 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964552    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964726 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964581    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:15.964726 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:15.964634    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065169 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065140    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065327 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-out\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065327 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065313    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065438 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065351    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065438 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065404    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065539 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p4fn\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-kube-api-access-8p4fn\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065539 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065505    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065634 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065534    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065634 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065561    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065634 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065590    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065634 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065851 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065650    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.065851 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.065679    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-web-config\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.067030 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:59:16.066130    2567 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 18 19:59:16.067030 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:59:16.066239    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-main-tls podName:0690f402-b706-4ecd-a2cc-2fdc08a51d3e nodeName:}" failed. No retries permitted until 2026-09-18 19:59:16.56621802 +0000 UTC m=+137.126801469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e") : secret "alertmanager-main-tls" not found
Sep 18 19:59:16.067030 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.066465    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.067030 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.066512    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.067030 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.066857    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.068992 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.068969    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.069451 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.069160    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.069451 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.069359    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-out\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.069451 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.069368    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.069755 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.069735    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.070174 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.070122    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-volume\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.070610 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.070583    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.070920 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.070900    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-web-config\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.085102 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.085073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p4fn\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-kube-api-access-8p4fn\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.388139 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.388102    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9c582" event={"ID":"0a56f260-3fc2-4202-ae5c-86461eb42d71","Type":"ContainerStarted","Data":"ecab7d432ee49846918e8481e0e9946430b3135df0683e8a6aeceeaa28a6b9e0"}
Sep 18 19:59:16.390977 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.390395    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66" event={"ID":"8a18ab65-fbb1-408f-b7f4-9168e5ee4847","Type":"ContainerStarted","Data":"c5014ff82c762e2cd8bd9c463079fab1e187ff9937c9bc08df3b2a84aa37df8f"}
Sep 18 19:59:16.570092 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.570059    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.572410 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.572385    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.742029 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.741951    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 19:59:16.788994 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.788966    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:16.789134 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.789031    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:16.794233 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:16.794206    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:17.187796 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.187765    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 19:59:17.190474 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:59:17.190437    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0690f402_b706_4ecd_a2cc_2fdc08a51d3e.slice/crio-d0ae37d8e6d60cf1c3455f39a107292d9d975ba4f6eb40a9677f5c38fa68281f WatchSource:0}: Error finding container d0ae37d8e6d60cf1c3455f39a107292d9d975ba4f6eb40a9677f5c38fa68281f: Status 404 returned error can't find the container with id d0ae37d8e6d60cf1c3455f39a107292d9d975ba4f6eb40a9677f5c38fa68281f
Sep 18 19:59:17.222008 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.221972    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-688c76f68d-56dm8" podUID="21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" containerName="registry" containerID="cri-o://ba98be91c1d8c4a73a5f4332dd02a1866f7dc1b1bc0533a890420f60fb820d20" gracePeriod=30
Sep 18 19:59:17.397184 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.397151    2567 generic.go:356] "Generic (PLEG): container finished" podID="0a56f260-3fc2-4202-ae5c-86461eb42d71" containerID="2f0b1b490d930cb7a512db3a0831ab81d1a3d2a493c296d021fd306e82439f8c" exitCode=0
Sep 18 19:59:17.397589 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.397354    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9c582" event={"ID":"0a56f260-3fc2-4202-ae5c-86461eb42d71","Type":"ContainerDied","Data":"2f0b1b490d930cb7a512db3a0831ab81d1a3d2a493c296d021fd306e82439f8c"}
Sep 18 19:59:17.399699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.399628    2567 generic.go:356] "Generic (PLEG): container finished" podID="21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" containerID="ba98be91c1d8c4a73a5f4332dd02a1866f7dc1b1bc0533a890420f60fb820d20" exitCode=0
Sep 18 19:59:17.399806 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.399760    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-688c76f68d-56dm8" event={"ID":"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa","Type":"ContainerDied","Data":"ba98be91c1d8c4a73a5f4332dd02a1866f7dc1b1bc0533a890420f60fb820d20"}
Sep 18 19:59:17.402007 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.401984    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66" event={"ID":"8a18ab65-fbb1-408f-b7f4-9168e5ee4847","Type":"ContainerStarted","Data":"8d58e83b8cd2686e5569d24def91b44e961bfefd850d747747860b8070e9557c"}
Sep 18 19:59:17.402109 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.402039    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66" event={"ID":"8a18ab65-fbb1-408f-b7f4-9168e5ee4847","Type":"ContainerStarted","Data":"e8c9969181439d10aee34971c512cd76900a7b305e8d3820ff63c0025cf04c0a"}
Sep 18 19:59:17.402109 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.402062    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66" event={"ID":"8a18ab65-fbb1-408f-b7f4-9168e5ee4847","Type":"ContainerStarted","Data":"1b702f5da0d831fbe075373487d40f3eb95ffc234c9ed61f74a2512ffb87be48"}
Sep 18 19:59:17.403693 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.403657    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerStarted","Data":"d0ae37d8e6d60cf1c3455f39a107292d9d975ba4f6eb40a9677f5c38fa68281f"}
Sep 18 19:59:17.408485 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.408351    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:17.474198 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.474138    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:59:17.484471 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.484421    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-whc66" podStartSLOduration=1.22697184 podStartE2EDuration="2.484402582s" totalImagesPullingTime="1.257430742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:59:17.483398858 +0000 UTC m=+138.043982314" watchObservedRunningTime="2026-09-18 19:59:17.484402582 +0000 UTC m=+138.044986040"
Sep 18 19:59:17.579494 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.579464    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-bound-sa-token\") pod \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") "
Sep 18 19:59:17.579679 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.579511    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k74zs\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-kube-api-access-k74zs\") pod \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") "
Sep 18 19:59:17.579679 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.579544    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-trusted-ca\") pod \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") "
Sep 18 19:59:17.579679 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.579577    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-installation-pull-secrets\") pod \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") "
Sep 18 19:59:17.579679 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.579623    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") pod \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") "
Sep 18 19:59:17.579679 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.579651    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-image-registry-private-configuration\") pod \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") "
Sep 18 19:59:17.580039 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.579711    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-ca-trust-extracted\") pod \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") "
Sep 18 19:59:17.580039 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.579774    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-certificates\") pod \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\" (UID: \"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa\") "
Sep 18 19:59:17.580039 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.580013    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:59:17.580528 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.580383    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-trusted-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:17.580528 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.580429    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:59:17.582331 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.582290    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:59:17.582904 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.582867    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-kube-api-access-k74zs" (OuterVolumeSpecName: "kube-api-access-k74zs") pod "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa"). InnerVolumeSpecName "kube-api-access-k74zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:59:17.583434 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.583411    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:59:17.583588 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.583563    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:59:17.584619 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.584597    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:59:17.591263 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.591217    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" (UID: "21c9fad7-16b9-4eac-a36e-9b6dbc2490fa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 19:59:17.681805 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.681778    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:17.681805 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.681804    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-image-registry-private-configuration\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:17.682008 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.681814    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-ca-trust-extracted\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:17.682008 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.681822    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-registry-certificates\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:17.682008 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.681831    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-bound-sa-token\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:17.682008 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.681839    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k74zs\" (UniqueName: \"kubernetes.io/projected/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-kube-api-access-k74zs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:17.682008 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:17.681847    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa-installation-pull-secrets\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:18.409398 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.409374    2567 generic.go:356] "Generic (PLEG): container finished" podID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerID="cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf" exitCode=0
Sep 18 19:59:18.409724 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.409471    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerDied","Data":"cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf"}
Sep 18 19:59:18.412039 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.412014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9c582" event={"ID":"0a56f260-3fc2-4202-ae5c-86461eb42d71","Type":"ContainerStarted","Data":"4532eb2a2739e3542c58568821deb3ab96f7c4f0801974bd84211cf957ac902f"}
Sep 18 19:59:18.412150 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.412070    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9c582" event={"ID":"0a56f260-3fc2-4202-ae5c-86461eb42d71","Type":"ContainerStarted","Data":"582017b36f048f65ff3905167dbfc5d0c6d2381fdbf75a64e189cd9fe49f6b7e"}
Sep 18 19:59:18.413413 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.413400    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-688c76f68d-56dm8"
Sep 18 19:59:18.413501 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.413430    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-688c76f68d-56dm8" event={"ID":"21c9fad7-16b9-4eac-a36e-9b6dbc2490fa","Type":"ContainerDied","Data":"750bc0d5430231a366ee6629ddf960b0b3ec47ad2ca31ce532e5881b0c004c8b"}
Sep 18 19:59:18.413501 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.413467    2567 scope.go:117] "RemoveContainer" containerID="ba98be91c1d8c4a73a5f4332dd02a1866f7dc1b1bc0533a890420f60fb820d20"
Sep 18 19:59:18.476772 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.476750    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-688c76f68d-56dm8"]
Sep 18 19:59:18.479625 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.479586    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-688c76f68d-56dm8"]
Sep 18 19:59:18.500329 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.500285    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9c582" podStartSLOduration=2.412249198 podStartE2EDuration="3.50027172s" totalImagesPullingTime="1.018938148s" totalInitContainerRuntime="69.084374ms" isStatefulPod=true podCreationTimestamp="2026-09-18 19:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:59:18.499167017 +0000 UTC m=+139.059750486" watchObservedRunningTime="2026-09-18 19:59:18.50027172 +0000 UTC m=+139.060855177"
Sep 18 19:59:18.637746 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.637720    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-79564b56cd-m2cm4"]
Sep 18 19:59:18.638001 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.637991    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" containerName="registry"
Sep 18 19:59:18.638044 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.638003    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" containerName="registry"
Sep 18 19:59:18.638077 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.638054    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" containerName="registry"
Sep 18 19:59:18.642499 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.642482    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.645246 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.645225    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 18 19:59:18.645369 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.645337    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 18 19:59:18.645466 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.645429    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-36vn10u1g9s2t\""
Sep 18 19:59:18.645539 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.645485    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hvg7l\""
Sep 18 19:59:18.645539 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.645508    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 18 19:59:18.645539 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.645525    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 18 19:59:18.645539 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.645507    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 18 19:59:18.654405 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.654387    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-79564b56cd-m2cm4"]
Sep 18 19:59:18.792264 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.792231    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.792264 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.792263    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-grpc-tls\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.792458 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.792289    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.792458 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.792319    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-tls\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.792458 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.792347    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.792458 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.792374    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqg9h\" (UniqueName: \"kubernetes.io/projected/853ddca3-13e7-4908-925e-7420e61ebd7d-kube-api-access-hqg9h\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.792593 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.792457    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.792593 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.792493    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/853ddca3-13e7-4908-925e-7420e61ebd7d-metrics-client-ca\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.893024 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.892959    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-tls\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.893024 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.892992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.893024 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.893010    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqg9h\" (UniqueName: \"kubernetes.io/projected/853ddca3-13e7-4908-925e-7420e61ebd7d-kube-api-access-hqg9h\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.893220 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.893055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.893220 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.893079    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/853ddca3-13e7-4908-925e-7420e61ebd7d-metrics-client-ca\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.893220 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.893110    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.893346 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.893227    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-grpc-tls\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.893346 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.893288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.893994 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.893966    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/853ddca3-13e7-4908-925e-7420e61ebd7d-metrics-client-ca\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.895514 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.895486    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.895699 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.895666    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.896037 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.896013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.896112 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.896037    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-tls\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.896112 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.896076    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-grpc-tls\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.896308 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.896294    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/853ddca3-13e7-4908-925e-7420e61ebd7d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.901561 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.901540    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqg9h\" (UniqueName: \"kubernetes.io/projected/853ddca3-13e7-4908-925e-7420e61ebd7d-kube-api-access-hqg9h\") pod \"thanos-querier-79564b56cd-m2cm4\" (UID: \"853ddca3-13e7-4908-925e-7420e61ebd7d\") " pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:18.951517 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:18.951498    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:19.088014 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.087983    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"]
Sep 18 19:59:19.091533 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.091514    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:19.094132 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.094092    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-dfdnc\""
Sep 18 19:59:19.094132 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.094109    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 18 19:59:19.114773 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.114738    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"]
Sep 18 19:59:19.195125 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.195061    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e69bd6f-7299-4297-94c1-9e962332bcf2-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-dgsmk\" (UID: \"3e69bd6f-7299-4297-94c1-9e962332bcf2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:19.195125 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.195098    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zhx2\" (UniqueName: \"kubernetes.io/projected/3e69bd6f-7299-4297-94c1-9e962332bcf2-kube-api-access-2zhx2\") pod \"monitoring-plugin-5984459ff8-dgsmk\" (UID: \"3e69bd6f-7299-4297-94c1-9e962332bcf2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:19.248251 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.248221    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-79564b56cd-m2cm4"]
Sep 18 19:59:19.251021 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:59:19.250990    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853ddca3_13e7_4908_925e_7420e61ebd7d.slice/crio-740d5535106cf9d79255c921085c53f037b2c722034cfec43187904857d0d988 WatchSource:0}: Error finding container 740d5535106cf9d79255c921085c53f037b2c722034cfec43187904857d0d988: Status 404 returned error can't find the container with id 740d5535106cf9d79255c921085c53f037b2c722034cfec43187904857d0d988
Sep 18 19:59:19.295712 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.295674    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e69bd6f-7299-4297-94c1-9e962332bcf2-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-dgsmk\" (UID: \"3e69bd6f-7299-4297-94c1-9e962332bcf2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:19.295818 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.295732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zhx2\" (UniqueName: \"kubernetes.io/projected/3e69bd6f-7299-4297-94c1-9e962332bcf2-kube-api-access-2zhx2\") pod \"monitoring-plugin-5984459ff8-dgsmk\" (UID: \"3e69bd6f-7299-4297-94c1-9e962332bcf2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:19.295890 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:59:19.295846    2567 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 18 19:59:19.295940 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:59:19.295917    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e69bd6f-7299-4297-94c1-9e962332bcf2-monitoring-plugin-cert podName:3e69bd6f-7299-4297-94c1-9e962332bcf2 nodeName:}" failed. No retries permitted until 2026-09-18 19:59:19.795895923 +0000 UTC m=+140.356479361 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3e69bd6f-7299-4297-94c1-9e962332bcf2-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-dgsmk" (UID: "3e69bd6f-7299-4297-94c1-9e962332bcf2") : secret "monitoring-plugin-cert" not found
Sep 18 19:59:19.310506 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.310483    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zhx2\" (UniqueName: \"kubernetes.io/projected/3e69bd6f-7299-4297-94c1-9e962332bcf2-kube-api-access-2zhx2\") pod \"monitoring-plugin-5984459ff8-dgsmk\" (UID: \"3e69bd6f-7299-4297-94c1-9e962332bcf2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:19.418922 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.418887    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4" event={"ID":"853ddca3-13e7-4908-925e-7420e61ebd7d","Type":"ContainerStarted","Data":"740d5535106cf9d79255c921085c53f037b2c722034cfec43187904857d0d988"}
Sep 18 19:59:19.557281 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.557208    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-658b6d7886-v5z8k"]
Sep 18 19:59:19.560761 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.560738    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.564191 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.564137    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 19:59:19.564320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.564233    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 18 19:59:19.564320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.564261    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-f6e77jcjsumo5\""
Sep 18 19:59:19.564320 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.564274    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 18 19:59:19.564494 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.564236    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fqxxk\""
Sep 18 19:59:19.564627 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.564606    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 18 19:59:19.574383 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.574363    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-658b6d7886-v5z8k"]
Sep 18 19:59:19.697723 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.697681    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21c9fad7-16b9-4eac-a36e-9b6dbc2490fa" path="/var/lib/kubelet/pods/21c9fad7-16b9-4eac-a36e-9b6dbc2490fa/volumes"
Sep 18 19:59:19.698969 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.698943    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6bdb\" (UniqueName: \"kubernetes.io/projected/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-kube-api-access-z6bdb\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.698969 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.698977    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-client-ca-bundle\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.699168 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.699003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-secret-metrics-server-tls\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.699168 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.699057    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-audit-log\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.699168 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.699092    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-secret-metrics-server-client-certs\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.699168 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.699109    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-metrics-server-audit-profiles\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.699168 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.699129    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.799633 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.799594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6bdb\" (UniqueName: \"kubernetes.io/projected/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-kube-api-access-z6bdb\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.799756 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.799642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-client-ca-bundle\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.799756 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.799661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-secret-metrics-server-tls\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.799756 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.799698    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e69bd6f-7299-4297-94c1-9e962332bcf2-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-dgsmk\" (UID: \"3e69bd6f-7299-4297-94c1-9e962332bcf2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:19.799756 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.799722    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-audit-log\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.800039 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.799758    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-secret-metrics-server-client-certs\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.800039 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.799788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-metrics-server-audit-profiles\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.800039 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.799816    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.800253 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.800222    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-audit-log\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.800565 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.800541    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.801276 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.801251    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-metrics-server-audit-profiles\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.802236 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.802213    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-secret-metrics-server-client-certs\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.802325 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.802298    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-secret-metrics-server-tls\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.802497 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.802476    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e69bd6f-7299-4297-94c1-9e962332bcf2-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-dgsmk\" (UID: \"3e69bd6f-7299-4297-94c1-9e962332bcf2\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:19.802580 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.802562    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-client-ca-bundle\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.815156 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.815134    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6bdb\" (UniqueName: \"kubernetes.io/projected/7e39f0d9-b2c3-4e34-aa29-e4215056c9cc-kube-api-access-z6bdb\") pod \"metrics-server-658b6d7886-v5z8k\" (UID: \"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc\") " pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:19.871900 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:19.871864    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:20.002840 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.002823    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:20.176525 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.176461    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-658b6d7886-v5z8k"]
Sep 18 19:59:20.178873 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:59:20.178840    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e39f0d9_b2c3_4e34_aa29_e4215056c9cc.slice/crio-1325721f116bb6e6c0142009f03a9909b88b0d6d23ccd459d924627f08ed7e9f WatchSource:0}: Error finding container 1325721f116bb6e6c0142009f03a9909b88b0d6d23ccd459d924627f08ed7e9f: Status 404 returned error can't find the container with id 1325721f116bb6e6c0142009f03a9909b88b0d6d23ccd459d924627f08ed7e9f
Sep 18 19:59:20.286793 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.286715    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"]
Sep 18 19:59:20.292851 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:59:20.292803    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e69bd6f_7299_4297_94c1_9e962332bcf2.slice/crio-ba3b56ddac831758e4a75412b7e38cf83d1eafb7f9ef8bb8c798ab27f594b204 WatchSource:0}: Error finding container ba3b56ddac831758e4a75412b7e38cf83d1eafb7f9ef8bb8c798ab27f594b204: Status 404 returned error can't find the container with id ba3b56ddac831758e4a75412b7e38cf83d1eafb7f9ef8bb8c798ab27f594b204
Sep 18 19:59:20.425866 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.425822    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk" event={"ID":"3e69bd6f-7299-4297-94c1-9e962332bcf2","Type":"ContainerStarted","Data":"ba3b56ddac831758e4a75412b7e38cf83d1eafb7f9ef8bb8c798ab27f594b204"}
Sep 18 19:59:20.427310 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.427247    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k" event={"ID":"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc","Type":"ContainerStarted","Data":"1325721f116bb6e6c0142009f03a9909b88b0d6d23ccd459d924627f08ed7e9f"}
Sep 18 19:59:20.430881 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.430851    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerStarted","Data":"9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879"}
Sep 18 19:59:20.430979 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.430921    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerStarted","Data":"39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e"}
Sep 18 19:59:20.430979 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.430962    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerStarted","Data":"6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1"}
Sep 18 19:59:20.431090 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.431003    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerStarted","Data":"bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909"}
Sep 18 19:59:20.431090 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:20.431041    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerStarted","Data":"276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f"}
Sep 18 19:59:21.034368 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.034332    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 19:59:21.038515 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.038489    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.043058 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.043035    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 18 19:59:21.043629 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.043586    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 18 19:59:21.044183 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.043960    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bzjsb\""
Sep 18 19:59:21.044183 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.043977    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 18 19:59:21.044338 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.044251    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 18 19:59:21.044338 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.044330    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 18 19:59:21.044503 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.044487    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 18 19:59:21.044569 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.044505    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 18 19:59:21.044627 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.044569    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 18 19:59:21.044627 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.044509    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 18 19:59:21.044746 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.044648    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 18 19:59:21.044793 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.044756    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 18 19:59:21.044870 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.044853    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-aps6ardtfvk6n\""
Sep 18 19:59:21.045642 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.045601    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 18 19:59:21.046730 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.046427    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 18 19:59:21.056763 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.056735    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 18 19:59:21.058948 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.058925    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 19:59:21.214626 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214587    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.214824 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214650    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.214824 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj6j2\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-kube-api-access-sj6j2\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.214824 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214779    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.214824 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214811    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215029 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214849    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215029 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214910    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215029 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214928    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215029 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214955    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215029 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214969    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215029 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.214989    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-config\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215029 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.215021    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215353 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.215072    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215353 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.215136    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215353 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.215159    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215353 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.215188    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215353 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.215207    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215353 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.215221    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215353 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.215238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.215353 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.215261    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316284 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316284 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316258    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316484 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316484 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316313    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316484 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316334    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316484 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316354    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316484 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316408    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316484 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316484 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316469    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj6j2\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-kube-api-access-sj6j2\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316527    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316562    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316611    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316710    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316750    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-config\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316786    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.316877 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.316847    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.318857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.317503    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.318857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.317796    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.318857 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.318242    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.319510 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.319367    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.319510 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.319375    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.320534 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.319810    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.320534 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.320103    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.320534 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.320341    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.323219 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.322797    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.323219 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.323042    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.323782 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.323727    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.323978 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.323939    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.323978 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.323957    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.324126 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.324103    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.324193 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.324150    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-config-out\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.324275 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.324245    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-config\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.324345 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.324285    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.324345 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.324327    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.324429 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.324412    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-web-config\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.329207 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.329186    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj6j2\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-kube-api-access-sj6j2\") pod \"prometheus-k8s-0\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:21.357621 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:21.357593    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:22.442235 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.442205    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k" event={"ID":"7e39f0d9-b2c3-4e34-aa29-e4215056c9cc","Type":"ContainerStarted","Data":"c7d91dc8359ac5b7c6c460a3fcca2c583bb4a4b58da23d30e9eb07bc798c2d59"}
Sep 18 19:59:22.444887 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.444861    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4" event={"ID":"853ddca3-13e7-4908-925e-7420e61ebd7d","Type":"ContainerStarted","Data":"9a769e5d3fa8e1792a75e1417a1d68ce8ffa884cb78f8c3c4555b984be6133b1"}
Sep 18 19:59:22.444960 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.444934    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4" event={"ID":"853ddca3-13e7-4908-925e-7420e61ebd7d","Type":"ContainerStarted","Data":"c14886d9ab7ccdc84dcd3ed4584c9a4c65cf89b877dcc71b5eced8169713984a"}
Sep 18 19:59:22.444993 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.444974    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4" event={"ID":"853ddca3-13e7-4908-925e-7420e61ebd7d","Type":"ContainerStarted","Data":"24bfa7ed0f39d953d1b2d74066343ca43322bdf53ae88e0756a14dff9bc04b59"}
Sep 18 19:59:22.445037 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.445006    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4" event={"ID":"853ddca3-13e7-4908-925e-7420e61ebd7d","Type":"ContainerStarted","Data":"bb8b8cde8d2cce963be64e840ca8b502fffd30358ac282ebb66ed5d63ca064c2"}
Sep 18 19:59:22.449142 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.449116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerStarted","Data":"270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7"}
Sep 18 19:59:22.450762 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.450736    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk" event={"ID":"3e69bd6f-7299-4297-94c1-9e962332bcf2","Type":"ContainerStarted","Data":"997c849773b9b7b3cd5f95f70391e722bcfdd4feb53c871012ab48094fc9ff7d"}
Sep 18 19:59:22.450969 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.450947    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:22.456328 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.456306    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk"
Sep 18 19:59:22.491838 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.491669    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k" podStartSLOduration=1.485206124 podStartE2EDuration="3.491653776s" totalImagesPullingTime="2.006447652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:59:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:59:22.487287337 +0000 UTC m=+143.047870793" watchObservedRunningTime="2026-09-18 19:59:22.491653776 +0000 UTC m=+143.052237233"
Sep 18 19:59:22.493112 ip-10-0-130-248 kubenswrapper[2567]: W0918 19:59:22.493089    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17177344_0e8b_4945_96f5_69259433ea2a.slice/crio-5a03c4d51c20ab259e81c54ef6f6e5f45708cca39de25729acc89ba61a3bfabb WatchSource:0}: Error finding container 5a03c4d51c20ab259e81c54ef6f6e5f45708cca39de25729acc89ba61a3bfabb: Status 404 returned error can't find the container with id 5a03c4d51c20ab259e81c54ef6f6e5f45708cca39de25729acc89ba61a3bfabb
Sep 18 19:59:22.493905 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.493885    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 19:59:22.574827 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.574624    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-dgsmk" podStartSLOduration=1.6184922899999998 podStartE2EDuration="3.574603108s" totalImagesPullingTime="1.956110818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:59:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:59:22.519511113 +0000 UTC m=+143.080094571" watchObservedRunningTime="2026-09-18 19:59:22.574603108 +0000 UTC m=+143.135186565"
Sep 18 19:59:22.575532 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:22.575486    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.040096685 podStartE2EDuration="7.575475828s" totalImagesPullingTime="4.516756341s" totalInitContainerRuntime="18.622802ms" isStatefulPod=false podCreationTimestamp="2026-09-18 19:59:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:59:22.571959515 +0000 UTC m=+143.132542975" watchObservedRunningTime="2026-09-18 19:59:22.575475828 +0000 UTC m=+143.136059282"
Sep 18 19:59:23.458273 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:23.458236    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4" event={"ID":"853ddca3-13e7-4908-925e-7420e61ebd7d","Type":"ContainerStarted","Data":"3729f0b16ebe627b3263ab4e7d7b21285d913b4d919c425f38524e3163125668"}
Sep 18 19:59:23.458719 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:23.458312    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4" event={"ID":"853ddca3-13e7-4908-925e-7420e61ebd7d","Type":"ContainerStarted","Data":"f21900c1a5af301a681a63dcc449c6a77195eccf0643b7458a61f9cbec0c7603"}
Sep 18 19:59:23.458719 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:23.458352    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:23.459910 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:23.459884    2567 generic.go:356] "Generic (PLEG): container finished" podID="17177344-0e8b-4945-96f5-69259433ea2a" containerID="8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15" exitCode=0
Sep 18 19:59:23.460011 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:23.459984    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerDied","Data":"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"}
Sep 18 19:59:23.460064 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:23.460053    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerStarted","Data":"5a03c4d51c20ab259e81c54ef6f6e5f45708cca39de25729acc89ba61a3bfabb"}
Sep 18 19:59:23.505353 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:23.505304    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4" podStartSLOduration=2.571349457 podStartE2EDuration="5.505283298s" totalImagesPullingTime="2.933933841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 19:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:59:23.501507267 +0000 UTC m=+144.062090736" watchObservedRunningTime="2026-09-18 19:59:23.505283298 +0000 UTC m=+144.065866755"
Sep 18 19:59:26.479906 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:26.479858    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerStarted","Data":"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"}
Sep 18 19:59:26.480226 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:26.479919    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerStarted","Data":"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"}
Sep 18 19:59:26.480226 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:26.479945    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerStarted","Data":"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"}
Sep 18 19:59:26.480226 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:26.479968    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerStarted","Data":"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"}
Sep 18 19:59:27.487157 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:27.487122    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerStarted","Data":"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"}
Sep 18 19:59:27.487577 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:27.487184    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerStarted","Data":"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"}
Sep 18 19:59:27.525029 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:27.524984    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.771633776 podStartE2EDuration="6.524969972s" totalImagesPullingTime="2.730389169s" totalInitContainerRuntime="22.947027ms" isStatefulPod=false podCreationTimestamp="2026-09-18 19:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 19:59:27.523807864 +0000 UTC m=+148.084391319" watchObservedRunningTime="2026-09-18 19:59:27.524969972 +0000 UTC m=+148.085553427"
Sep 18 19:59:29.471837 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:29.471810    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-79564b56cd-m2cm4"
Sep 18 19:59:31.357862 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:31.357820    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 19:59:33.326274 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:33.326242    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-694794746c-6s9q5"]
Sep 18 19:59:35.526787 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:35.526755    2567 generic.go:356] "Generic (PLEG): container finished" podID="3c77ae59-6d5b-4d9e-a223-4394d7df9656" containerID="a8341aa347b13a5d0337c5af16a7dab23a7dd3c73ff07738a308a91614843857" exitCode=0
Sep 18 19:59:35.527082 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:35.526823    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls" event={"ID":"3c77ae59-6d5b-4d9e-a223-4394d7df9656","Type":"ContainerDied","Data":"a8341aa347b13a5d0337c5af16a7dab23a7dd3c73ff07738a308a91614843857"}
Sep 18 19:59:35.527144 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:35.527115    2567 scope.go:117] "RemoveContainer" containerID="a8341aa347b13a5d0337c5af16a7dab23a7dd3c73ff07738a308a91614843857"
Sep 18 19:59:36.533101 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:36.533070    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-d7dls" event={"ID":"3c77ae59-6d5b-4d9e-a223-4394d7df9656","Type":"ContainerStarted","Data":"13dd3752985e3e1b490e33445c28cab964731462c3fb51246ea83db9049ba9cc"}
Sep 18 19:59:39.872777 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:39.872746    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:39.873140 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:39.872792    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:44.568458 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:44.568427    2567 generic.go:356] "Generic (PLEG): container finished" podID="f5be3994-b3be-4530-bdde-d207770b129b" containerID="f92e4385e315f7d38f167b43319525a1b19dfc7b1bdbc916a8326950e8a8d3a0" exitCode=0
Sep 18 19:59:44.568795 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:44.568498    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm" event={"ID":"f5be3994-b3be-4530-bdde-d207770b129b","Type":"ContainerDied","Data":"f92e4385e315f7d38f167b43319525a1b19dfc7b1bdbc916a8326950e8a8d3a0"}
Sep 18 19:59:44.568901 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:44.568884    2567 scope.go:117] "RemoveContainer" containerID="f92e4385e315f7d38f167b43319525a1b19dfc7b1bdbc916a8326950e8a8d3a0"
Sep 18 19:59:45.574443 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:45.574406    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-2npsm" event={"ID":"f5be3994-b3be-4530-bdde-d207770b129b","Type":"ContainerStarted","Data":"d17cf5d2b55eadc57486b3365182ddbf11b53a112db2daffd822ca18527d3371"}
Sep 18 19:59:58.345535 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.345470    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-694794746c-6s9q5" podUID="22b10eae-315c-4097-8f35-345c43b51b5d" containerName="console" containerID="cri-o://92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb" gracePeriod=15
Sep 18 19:59:58.623709 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.623670    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:58.630992 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.630960    2567 generic.go:356] "Generic (PLEG): container finished" podID="22b10eae-315c-4097-8f35-345c43b51b5d" containerID="92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb" exitCode=0
Sep 18 19:59:58.631121 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.631001    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-694794746c-6s9q5"
Sep 18 19:59:58.631121 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.631027    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694794746c-6s9q5" event={"ID":"22b10eae-315c-4097-8f35-345c43b51b5d","Type":"ContainerDied","Data":"92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb"}
Sep 18 19:59:58.631121 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.631081    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-694794746c-6s9q5" event={"ID":"22b10eae-315c-4097-8f35-345c43b51b5d","Type":"ContainerDied","Data":"2f3b3f0fa8b35cd2870a01748bf2069ce25155cd0de61dd40ed93522192e9efd"}
Sep 18 19:59:58.631121 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.631108    2567 scope.go:117] "RemoveContainer" containerID="92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb"
Sep 18 19:59:58.641098 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.641082    2567 scope.go:117] "RemoveContainer" containerID="92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb"
Sep 18 19:59:58.641396 ip-10-0-130-248 kubenswrapper[2567]: E0918 19:59:58.641370    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb\": container with ID starting with 92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb not found: ID does not exist" containerID="92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb"
Sep 18 19:59:58.641483 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.641406    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb"} err="failed to get container status \"92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb\": rpc error: code = NotFound desc = could not find container \"92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb\": container with ID starting with 92cd33914d9269adb7839c0f638b2b6b97c7dad8442c01682dcf7395fed1e6eb not found: ID does not exist"
Sep 18 19:59:58.725993 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.725965    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-serving-cert\") pod \"22b10eae-315c-4097-8f35-345c43b51b5d\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") "
Sep 18 19:59:58.726160 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726011    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-oauth-serving-cert\") pod \"22b10eae-315c-4097-8f35-345c43b51b5d\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") "
Sep 18 19:59:58.726160 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726047    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-trusted-ca-bundle\") pod \"22b10eae-315c-4097-8f35-345c43b51b5d\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") "
Sep 18 19:59:58.726270 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726171    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-console-config\") pod \"22b10eae-315c-4097-8f35-345c43b51b5d\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") "
Sep 18 19:59:58.726270 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726241    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-oauth-config\") pod \"22b10eae-315c-4097-8f35-345c43b51b5d\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") "
Sep 18 19:59:58.726500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726271    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-service-ca\") pod \"22b10eae-315c-4097-8f35-345c43b51b5d\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") "
Sep 18 19:59:58.726500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726300    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tpzg\" (UniqueName: \"kubernetes.io/projected/22b10eae-315c-4097-8f35-345c43b51b5d-kube-api-access-7tpzg\") pod \"22b10eae-315c-4097-8f35-345c43b51b5d\" (UID: \"22b10eae-315c-4097-8f35-345c43b51b5d\") "
Sep 18 19:59:58.726500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726375    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "22b10eae-315c-4097-8f35-345c43b51b5d" (UID: "22b10eae-315c-4097-8f35-345c43b51b5d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:59:58.726500 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726389    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "22b10eae-315c-4097-8f35-345c43b51b5d" (UID: "22b10eae-315c-4097-8f35-345c43b51b5d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:59:58.726720 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726562    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-console-config" (OuterVolumeSpecName: "console-config") pod "22b10eae-315c-4097-8f35-345c43b51b5d" (UID: "22b10eae-315c-4097-8f35-345c43b51b5d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:59:58.726720 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726601    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-service-ca" (OuterVolumeSpecName: "service-ca") pod "22b10eae-315c-4097-8f35-345c43b51b5d" (UID: "22b10eae-315c-4097-8f35-345c43b51b5d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 19:59:58.726720 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726611    2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-oauth-serving-cert\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:58.726720 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.726648    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-trusted-ca-bundle\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:58.728371 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.728341    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "22b10eae-315c-4097-8f35-345c43b51b5d" (UID: "22b10eae-315c-4097-8f35-345c43b51b5d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:59:58.728751 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.728726    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "22b10eae-315c-4097-8f35-345c43b51b5d" (UID: "22b10eae-315c-4097-8f35-345c43b51b5d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 19:59:58.728863 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.728814    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22b10eae-315c-4097-8f35-345c43b51b5d-kube-api-access-7tpzg" (OuterVolumeSpecName: "kube-api-access-7tpzg") pod "22b10eae-315c-4097-8f35-345c43b51b5d" (UID: "22b10eae-315c-4097-8f35-345c43b51b5d"). InnerVolumeSpecName "kube-api-access-7tpzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 19:59:58.827329 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.827300    2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-serving-cert\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:58.827329 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.827327    2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-console-config\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:58.827329 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.827335    2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/22b10eae-315c-4097-8f35-345c43b51b5d-console-oauth-config\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:58.827544 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.827343    2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/22b10eae-315c-4097-8f35-345c43b51b5d-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:58.827544 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.827352    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tpzg\" (UniqueName: \"kubernetes.io/projected/22b10eae-315c-4097-8f35-345c43b51b5d-kube-api-access-7tpzg\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 19:59:58.959313 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.959288    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-694794746c-6s9q5"]
Sep 18 19:59:58.960580 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:58.960558    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-694794746c-6s9q5"]
Sep 18 19:59:59.697489 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:59.697459    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22b10eae-315c-4097-8f35-345c43b51b5d" path="/var/lib/kubelet/pods/22b10eae-315c-4097-8f35-345c43b51b5d/volumes"
Sep 18 19:59:59.877423 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:59.877399    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 19:59:59.881214 ip-10-0-130-248 kubenswrapper[2567]: I0918 19:59:59.881183    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-658b6d7886-v5z8k"
Sep 18 20:00:21.358238 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:21.358191    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:21.377649 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:21.377623    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:21.739976 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:21.739905    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:35.204096 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.204056    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 20:00:35.204503 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.204379    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="alertmanager" containerID="cri-o://276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f" gracePeriod=120
Sep 18 20:00:35.204503 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.204422    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="prom-label-proxy" containerID="cri-o://270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7" gracePeriod=120
Sep 18 20:00:35.204503 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.204446    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy-web" containerID="cri-o://6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1" gracePeriod=120
Sep 18 20:00:35.204646 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.204462    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="config-reloader" containerID="cri-o://bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909" gracePeriod=120
Sep 18 20:00:35.204646 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.204379    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy-metric" containerID="cri-o://9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879" gracePeriod=120
Sep 18 20:00:35.204766 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.204460    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy" containerID="cri-o://39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e" gracePeriod=120
Sep 18 20:00:35.783967 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.783939    2567 generic.go:356] "Generic (PLEG): container finished" podID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerID="270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7" exitCode=0
Sep 18 20:00:35.783967 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.783962    2567 generic.go:356] "Generic (PLEG): container finished" podID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerID="39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e" exitCode=0
Sep 18 20:00:35.783967 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.783970    2567 generic.go:356] "Generic (PLEG): container finished" podID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerID="bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909" exitCode=0
Sep 18 20:00:35.783967 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.783977    2567 generic.go:356] "Generic (PLEG): container finished" podID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerID="276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f" exitCode=0
Sep 18 20:00:35.784218 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.784017    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerDied","Data":"270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7"}
Sep 18 20:00:35.784218 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.784084    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerDied","Data":"39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e"}
Sep 18 20:00:35.784218 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.784111    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerDied","Data":"bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909"}
Sep 18 20:00:35.784218 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:35.784135    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerDied","Data":"276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f"}
Sep 18 20:00:36.449118 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.449094    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.511532 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511504    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511532 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511537    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-out\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511737 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511557    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-volume\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511737 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511597    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-main-db\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511737 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511621    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511737 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511673    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-cluster-tls-config\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511737 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511733    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-main-tls\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511781    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-trusted-ca-bundle\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511827    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-metrics-client-ca\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511857    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-web-config\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511882    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p4fn\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-kube-api-access-8p4fn\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.511998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.511904    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-web\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.512906 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.512245    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:00:36.512906 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.512307    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-tls-assets\") pod \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\" (UID: \"0690f402-b706-4ecd-a2cc-2fdc08a51d3e\") "
Sep 18 20:00:36.512906 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.512590    2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.513567 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.513537    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:00:36.513716 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.513673    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:00:36.514610 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.514582    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:36.515497 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.515472    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:36.515587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.515514    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:36.515587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.515524    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-out" (OuterVolumeSpecName: "config-out") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:00:36.516194 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.516127    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-kube-api-access-8p4fn" (OuterVolumeSpecName: "kube-api-access-8p4fn") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "kube-api-access-8p4fn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:00:36.516323 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.516303    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-volume" (OuterVolumeSpecName: "config-volume") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:36.516533 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.516506    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:36.516803 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.516780    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:00:36.519406 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.519380    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:36.527359 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.527336    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-web-config" (OuterVolumeSpecName: "web-config") pod "0690f402-b706-4ecd-a2cc-2fdc08a51d3e" (UID: "0690f402-b706-4ecd-a2cc-2fdc08a51d3e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:36.612970 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.612947    2567 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-cluster-tls-config\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.612970 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.612971    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-main-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.612980    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-metrics-client-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.613006    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-web-config\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.613019    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p4fn\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-kube-api-access-8p4fn\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.613033    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.613042    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-tls-assets\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.613050    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.613059    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-out\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.613067    2567 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.613074    2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-alertmanager-main-db\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.613104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.613081    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0690f402-b706-4ecd-a2cc-2fdc08a51d3e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:36.791038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.790960    2567 generic.go:356] "Generic (PLEG): container finished" podID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerID="9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879" exitCode=0
Sep 18 20:00:36.791038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.790984    2567 generic.go:356] "Generic (PLEG): container finished" podID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerID="6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1" exitCode=0
Sep 18 20:00:36.791038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.791017    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerDied","Data":"9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879"}
Sep 18 20:00:36.791297 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.791049    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.791297 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.791074    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerDied","Data":"6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1"}
Sep 18 20:00:36.791297 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.791101    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0690f402-b706-4ecd-a2cc-2fdc08a51d3e","Type":"ContainerDied","Data":"d0ae37d8e6d60cf1c3455f39a107292d9d975ba4f6eb40a9677f5c38fa68281f"}
Sep 18 20:00:36.791297 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.791134    2567 scope.go:117] "RemoveContainer" containerID="270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7"
Sep 18 20:00:36.799584 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.799564    2567 scope.go:117] "RemoveContainer" containerID="9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879"
Sep 18 20:00:36.806303 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.806287    2567 scope.go:117] "RemoveContainer" containerID="39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e"
Sep 18 20:00:36.812743 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.812718    2567 scope.go:117] "RemoveContainer" containerID="6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1"
Sep 18 20:00:36.816729 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.816707    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 20:00:36.820246 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.820224    2567 scope.go:117] "RemoveContainer" containerID="bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909"
Sep 18 20:00:36.820725 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.820673    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 20:00:36.826601 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.826582    2567 scope.go:117] "RemoveContainer" containerID="276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f"
Sep 18 20:00:36.833567 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.833547    2567 scope.go:117] "RemoveContainer" containerID="cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf"
Sep 18 20:00:36.839946 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.839924    2567 scope.go:117] "RemoveContainer" containerID="270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7"
Sep 18 20:00:36.840179 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:36.840161    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7\": container with ID starting with 270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7 not found: ID does not exist" containerID="270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7"
Sep 18 20:00:36.840222 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.840186    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7"} err="failed to get container status \"270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7\": rpc error: code = NotFound desc = could not find container \"270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7\": container with ID starting with 270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7 not found: ID does not exist"
Sep 18 20:00:36.840222 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.840201    2567 scope.go:117] "RemoveContainer" containerID="9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879"
Sep 18 20:00:36.840441 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:36.840424    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879\": container with ID starting with 9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879 not found: ID does not exist" containerID="9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879"
Sep 18 20:00:36.840478 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.840448    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879"} err="failed to get container status \"9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879\": rpc error: code = NotFound desc = could not find container \"9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879\": container with ID starting with 9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879 not found: ID does not exist"
Sep 18 20:00:36.840478 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.840463    2567 scope.go:117] "RemoveContainer" containerID="39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e"
Sep 18 20:00:36.840710 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:36.840672    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e\": container with ID starting with 39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e not found: ID does not exist" containerID="39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e"
Sep 18 20:00:36.840785 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.840711    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e"} err="failed to get container status \"39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e\": rpc error: code = NotFound desc = could not find container \"39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e\": container with ID starting with 39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e not found: ID does not exist"
Sep 18 20:00:36.840785 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.840733    2567 scope.go:117] "RemoveContainer" containerID="6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1"
Sep 18 20:00:36.841109 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:36.841088    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1\": container with ID starting with 6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1 not found: ID does not exist" containerID="6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1"
Sep 18 20:00:36.841189 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.841115    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1"} err="failed to get container status \"6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1\": rpc error: code = NotFound desc = could not find container \"6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1\": container with ID starting with 6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1 not found: ID does not exist"
Sep 18 20:00:36.841189 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.841134    2567 scope.go:117] "RemoveContainer" containerID="bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909"
Sep 18 20:00:36.841503 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:36.841478    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909\": container with ID starting with bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909 not found: ID does not exist" containerID="bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909"
Sep 18 20:00:36.841661 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.841508    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909"} err="failed to get container status \"bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909\": rpc error: code = NotFound desc = could not find container \"bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909\": container with ID starting with bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909 not found: ID does not exist"
Sep 18 20:00:36.841661 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.841529    2567 scope.go:117] "RemoveContainer" containerID="276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f"
Sep 18 20:00:36.841870 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:36.841852    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f\": container with ID starting with 276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f not found: ID does not exist" containerID="276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f"
Sep 18 20:00:36.841931 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.841887    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f"} err="failed to get container status \"276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f\": rpc error: code = NotFound desc = could not find container \"276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f\": container with ID starting with 276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f not found: ID does not exist"
Sep 18 20:00:36.841931 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.841902    2567 scope.go:117] "RemoveContainer" containerID="cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf"
Sep 18 20:00:36.842234 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:36.842206    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf\": container with ID starting with cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf not found: ID does not exist" containerID="cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf"
Sep 18 20:00:36.842309 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.842237    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf"} err="failed to get container status \"cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf\": rpc error: code = NotFound desc = could not find container \"cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf\": container with ID starting with cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf not found: ID does not exist"
Sep 18 20:00:36.842309 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.842254    2567 scope.go:117] "RemoveContainer" containerID="270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7"
Sep 18 20:00:36.842526 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.842507    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7"} err="failed to get container status \"270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7\": rpc error: code = NotFound desc = could not find container \"270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7\": container with ID starting with 270f51d115a5809fdf1496e234da706d7e18fca958c1e1a298e0dcf5863372a7 not found: ID does not exist"
Sep 18 20:00:36.842573 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.842526    2567 scope.go:117] "RemoveContainer" containerID="9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879"
Sep 18 20:00:36.842792 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.842775    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879"} err="failed to get container status \"9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879\": rpc error: code = NotFound desc = could not find container \"9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879\": container with ID starting with 9bd62578d8005de2461ed624a6c7768c041bc332d8880a42e23e32fbb0994879 not found: ID does not exist"
Sep 18 20:00:36.842850 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.842792    2567 scope.go:117] "RemoveContainer" containerID="39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e"
Sep 18 20:00:36.843021 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.843006    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e"} err="failed to get container status \"39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e\": rpc error: code = NotFound desc = could not find container \"39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e\": container with ID starting with 39e14a2e2df882d801844111b5564c73772e6275aefb525d71a2dba00cbb441e not found: ID does not exist"
Sep 18 20:00:36.843068 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.843022    2567 scope.go:117] "RemoveContainer" containerID="6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1"
Sep 18 20:00:36.843263 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.843234    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1"} err="failed to get container status \"6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1\": rpc error: code = NotFound desc = could not find container \"6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1\": container with ID starting with 6fdfa2d42408ecf41881866f99e98a2fb7b47fbe25d0356a61da909c6d0d46a1 not found: ID does not exist"
Sep 18 20:00:36.843263 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.843255    2567 scope.go:117] "RemoveContainer" containerID="bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909"
Sep 18 20:00:36.843474 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.843459    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909"} err="failed to get container status \"bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909\": rpc error: code = NotFound desc = could not find container \"bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909\": container with ID starting with bf68d292c1b96e119701e18139ba552e9539ae77c5a7ed1a0b68fbf7fd83e909 not found: ID does not exist"
Sep 18 20:00:36.843531 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.843475    2567 scope.go:117] "RemoveContainer" containerID="276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f"
Sep 18 20:00:36.843735 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.843709    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f"} err="failed to get container status \"276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f\": rpc error: code = NotFound desc = could not find container \"276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f\": container with ID starting with 276c19de64c162813293da918a51ee7254db1c719b21054be20ee7d9bda1d82f not found: ID does not exist"
Sep 18 20:00:36.843793 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.843736    2567 scope.go:117] "RemoveContainer" containerID="cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf"
Sep 18 20:00:36.843940 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.843919    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf"} err="failed to get container status \"cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf\": rpc error: code = NotFound desc = could not find container \"cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf\": container with ID starting with cfc0b02f369cc27ac04bc90bf0829710bc8dfcb5a3752cb28b8cbbc49a36fccf not found: ID does not exist"
Sep 18 20:00:36.855643 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.855623    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 20:00:36.855944 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.855930    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy"
Sep 18 20:00:36.855993 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.855945    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy"
Sep 18 20:00:36.855993 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.855956    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy-web"
Sep 18 20:00:36.855993 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.855962    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy-web"
Sep 18 20:00:36.855993 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.855971    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22b10eae-315c-4097-8f35-345c43b51b5d" containerName="console"
Sep 18 20:00:36.855993 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.855976    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="22b10eae-315c-4097-8f35-345c43b51b5d" containerName="console"
Sep 18 20:00:36.855993 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.855984    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="alertmanager"
Sep 18 20:00:36.855993 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.855992    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="alertmanager"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856002    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="init-config-reloader"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856007    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="init-config-reloader"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856012    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="prom-label-proxy"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856017    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="prom-label-proxy"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856024    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="config-reloader"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856028    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="config-reloader"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856037    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy-metric"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856043    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy-metric"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856096    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy-web"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856104    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="config-reloader"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856110    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="22b10eae-315c-4097-8f35-345c43b51b5d" containerName="console"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856116    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="alertmanager"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856121    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856126    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="kube-rbac-proxy-metric"
Sep 18 20:00:36.856185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.856132    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" containerName="prom-label-proxy"
Sep 18 20:00:36.861127 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.861111    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.865355 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.865337    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 20:00:36.865454 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.865411    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 20:00:36.865583 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.865560    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 20:00:36.865667 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.865612    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 20:00:36.865667 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.865626    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 20:00:36.865784 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.865612    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 20:00:36.865784 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.865678    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 20:00:36.866133 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.866112    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tp2nk\""
Sep 18 20:00:36.866284 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.866256    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 20:00:36.874905 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.874863    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 20:00:36.876666 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.876643    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 20:00:36.914996 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.914972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-web-config\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915114 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915007    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-config-volume\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915114 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915033    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fce943-a9ea-4626-8496-f43fbb7787b1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915114 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915061    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9fce943-a9ea-4626-8496-f43fbb7787b1-config-out\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915114 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9fce943-a9ea-4626-8496-f43fbb7787b1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915283 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915113    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhg45\" (UniqueName: \"kubernetes.io/projected/e9fce943-a9ea-4626-8496-f43fbb7787b1-kube-api-access-dhg45\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915283 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915199    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915283 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915234    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915283 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915252    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9fce943-a9ea-4626-8496-f43fbb7787b1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915283 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915434 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915332    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915434 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915415    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:36.915498 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:36.915459    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9fce943-a9ea-4626-8496-f43fbb7787b1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.016738 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.016712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.016827 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.016753    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.016827 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.016779    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9fce943-a9ea-4626-8496-f43fbb7787b1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.016906 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.016825    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-web-config\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.016906 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.016848    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-config-volume\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.016906 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.016874    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fce943-a9ea-4626-8496-f43fbb7787b1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.016906 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.016902    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9fce943-a9ea-4626-8496-f43fbb7787b1-config-out\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.017083 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.016921    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9fce943-a9ea-4626-8496-f43fbb7787b1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.017083 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.017012    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhg45\" (UniqueName: \"kubernetes.io/projected/e9fce943-a9ea-4626-8496-f43fbb7787b1-kube-api-access-dhg45\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.017178 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.017080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.017178 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.017117    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.017178 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.017165    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9fce943-a9ea-4626-8496-f43fbb7787b1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.017379 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.017194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.017706 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.017667    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e9fce943-a9ea-4626-8496-f43fbb7787b1-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.017859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.017820    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9fce943-a9ea-4626-8496-f43fbb7787b1-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.019873 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.019845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-web-config\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.019873 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.019845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.020021 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.019901    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e9fce943-a9ea-4626-8496-f43fbb7787b1-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.020021 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.019917    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-config-volume\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.020021 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.020014    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.020171 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.020120    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.020171 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.020137    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e9fce943-a9ea-4626-8496-f43fbb7787b1-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.020234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.020190    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e9fce943-a9ea-4626-8496-f43fbb7787b1-config-out\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.021001 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.020980    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.021893 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.021877    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e9fce943-a9ea-4626-8496-f43fbb7787b1-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.028503 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.028486    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhg45\" (UniqueName: \"kubernetes.io/projected/e9fce943-a9ea-4626-8496-f43fbb7787b1-kube-api-access-dhg45\") pod \"alertmanager-main-0\" (UID: \"e9fce943-a9ea-4626-8496-f43fbb7787b1\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.175520 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.175486    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 20:00:37.461302 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.461276    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 20:00:37.463286 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:00:37.463265    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fce943_a9ea_4626_8496_f43fbb7787b1.slice/crio-a3ef0c0be5968fd5ccd8a0de3824098da3752ce34327e3d38022fd7d930e2c70 WatchSource:0}: Error finding container a3ef0c0be5968fd5ccd8a0de3824098da3752ce34327e3d38022fd7d930e2c70: Status 404 returned error can't find the container with id a3ef0c0be5968fd5ccd8a0de3824098da3752ce34327e3d38022fd7d930e2c70
Sep 18 20:00:37.697761 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.697732    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0690f402-b706-4ecd-a2cc-2fdc08a51d3e" path="/var/lib/kubelet/pods/0690f402-b706-4ecd-a2cc-2fdc08a51d3e/volumes"
Sep 18 20:00:37.796500 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.796437    2567 generic.go:356] "Generic (PLEG): container finished" podID="e9fce943-a9ea-4626-8496-f43fbb7787b1" containerID="ef123d7dd10eeeedff6e1440fe4b6840590786bd54758ead41d4dde17f42079e" exitCode=0
Sep 18 20:00:37.796597 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.796520    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9fce943-a9ea-4626-8496-f43fbb7787b1","Type":"ContainerDied","Data":"ef123d7dd10eeeedff6e1440fe4b6840590786bd54758ead41d4dde17f42079e"}
Sep 18 20:00:37.796639 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:37.796593    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9fce943-a9ea-4626-8496-f43fbb7787b1","Type":"ContainerStarted","Data":"a3ef0c0be5968fd5ccd8a0de3824098da3752ce34327e3d38022fd7d930e2c70"}
Sep 18 20:00:38.804799 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:38.804766    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9fce943-a9ea-4626-8496-f43fbb7787b1","Type":"ContainerStarted","Data":"2c35e871f9ebf1a0f4a7623a551bce19b680e2ee06c8045be7af72fc6a46cd22"}
Sep 18 20:00:38.805223 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:38.804854    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9fce943-a9ea-4626-8496-f43fbb7787b1","Type":"ContainerStarted","Data":"8436f37fdd0385093e27d89d2d2588c8694258a5c1e1359be5fe5fe38dd98529"}
Sep 18 20:00:38.805223 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:38.804878    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9fce943-a9ea-4626-8496-f43fbb7787b1","Type":"ContainerStarted","Data":"88c90661b902838c22fe8e68fc626d884ed00c4e4fa3ee87923f13addf680eae"}
Sep 18 20:00:38.805223 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:38.804901    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9fce943-a9ea-4626-8496-f43fbb7787b1","Type":"ContainerStarted","Data":"37f61916a1ef626b9d1cd526df9f37d0a454d012d34e8ab719b0333a2d31fdfb"}
Sep 18 20:00:38.805223 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:38.804922    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9fce943-a9ea-4626-8496-f43fbb7787b1","Type":"ContainerStarted","Data":"fb7a134bbbbb354242e7385d29e28bbc27f0d501d874ee2382b72e1c913c6e40"}
Sep 18 20:00:38.805223 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:38.804947    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e9fce943-a9ea-4626-8496-f43fbb7787b1","Type":"ContainerStarted","Data":"b5d9dcd8f2fddf8712e42d95d697cbf6ac117201078832ee9279c030cf4466ea"}
Sep 18 20:00:38.836934 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:38.836891    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8179080770000002 podStartE2EDuration="2.836875961s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.967884ms" isStatefulPod=false podCreationTimestamp="2026-09-18 20:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:00:38.835060905 +0000 UTC m=+219.395644386" watchObservedRunningTime="2026-09-18 20:00:38.836875961 +0000 UTC m=+219.397459416"
Sep 18 20:00:39.536949 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.536916    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 20:00:39.537299 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.537264    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="prometheus" containerID="cri-o://288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff" gracePeriod=600
Sep 18 20:00:39.537464 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.537296    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="thanos-sidecar" containerID="cri-o://aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c" gracePeriod=600
Sep 18 20:00:39.537464 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.537319    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy-web" containerID="cri-o://9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15" gracePeriod=600
Sep 18 20:00:39.537464 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.537292    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456" gracePeriod=600
Sep 18 20:00:39.537464 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.537403    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy" containerID="cri-o://53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5" gracePeriod=600
Sep 18 20:00:39.537464 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.537307    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="config-reloader" containerID="cri-o://feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02" gracePeriod=600
Sep 18 20:00:39.774542 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.774518    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:39.813739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813637    2567 generic.go:356] "Generic (PLEG): container finished" podID="17177344-0e8b-4945-96f5-69259433ea2a" containerID="c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456" exitCode=0
Sep 18 20:00:39.813739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813668    2567 generic.go:356] "Generic (PLEG): container finished" podID="17177344-0e8b-4945-96f5-69259433ea2a" containerID="53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5" exitCode=0
Sep 18 20:00:39.813739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813679    2567 generic.go:356] "Generic (PLEG): container finished" podID="17177344-0e8b-4945-96f5-69259433ea2a" containerID="9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15" exitCode=0
Sep 18 20:00:39.813739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813717    2567 generic.go:356] "Generic (PLEG): container finished" podID="17177344-0e8b-4945-96f5-69259433ea2a" containerID="aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c" exitCode=0
Sep 18 20:00:39.813739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813724    2567 generic.go:356] "Generic (PLEG): container finished" podID="17177344-0e8b-4945-96f5-69259433ea2a" containerID="feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02" exitCode=0
Sep 18 20:00:39.813739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813733    2567 generic.go:356] "Generic (PLEG): container finished" podID="17177344-0e8b-4945-96f5-69259433ea2a" containerID="288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff" exitCode=0
Sep 18 20:00:39.814242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813755    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:39.814242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813720    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerDied","Data":"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"}
Sep 18 20:00:39.814242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerDied","Data":"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"}
Sep 18 20:00:39.814242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813951    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerDied","Data":"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"}
Sep 18 20:00:39.814242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813974    2567 scope.go:117] "RemoveContainer" containerID="c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"
Sep 18 20:00:39.814242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.813991    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerDied","Data":"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"}
Sep 18 20:00:39.814242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.814030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerDied","Data":"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"}
Sep 18 20:00:39.814242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.814070    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerDied","Data":"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"}
Sep 18 20:00:39.814242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.814105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"17177344-0e8b-4945-96f5-69259433ea2a","Type":"ContainerDied","Data":"5a03c4d51c20ab259e81c54ef6f6e5f45708cca39de25729acc89ba61a3bfabb"}
Sep 18 20:00:39.823174 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.823134    2567 scope.go:117] "RemoveContainer" containerID="53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"
Sep 18 20:00:39.830707 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.830673    2567 scope.go:117] "RemoveContainer" containerID="9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"
Sep 18 20:00:39.838239 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.838213    2567 scope.go:117] "RemoveContainer" containerID="aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"
Sep 18 20:00:39.843393 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843361    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-grpc-tls\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843495 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843403    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-kubelet-serving-ca-bundle\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843495 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843426    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-tls\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843495 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843450    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-metrics-client-certs\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843495 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843478    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-config\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843720 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843606    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-web-config\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843720 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843651    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-2\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843720 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843705    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sj6j2\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-kube-api-access-sj6j2\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843737    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-1\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843761    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-kube-rbac-proxy\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843831    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843857    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-serving-certs-ca-bundle\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843883    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-thanos-prometheus-http-client-file\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843916    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-metrics-client-ca\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843950    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-0\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.843984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.843981    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-tls-assets\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.844409 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.844011    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-config-out\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.844409 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.844035    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-db\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.844409 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.844066    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.844409 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.844098    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-trusted-ca-bundle\") pod \"17177344-0e8b-4945-96f5-69259433ea2a\" (UID: \"17177344-0e8b-4945-96f5-69259433ea2a\") "
Sep 18 20:00:39.845022 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.844956    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:00:39.845022 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.844991    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:00:39.846199 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.845417    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:00:39.846199 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.845481    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:00:39.846931 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.846896    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:39.846931 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.846903    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:00:39.847137 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.846955    2567 scope.go:117] "RemoveContainer" containerID="feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"
Sep 18 20:00:39.847579 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.847552    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:00:39.847664 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.847611    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:00:39.848618 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.848567    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-config" (OuterVolumeSpecName: "config") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:39.848618 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.848594    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:39.848925 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.848713    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:00:39.849038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.849011    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:39.849087 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.849041    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:00:39.849509 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.849472    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:39.849599 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.849546    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-kube-api-access-sj6j2" (OuterVolumeSpecName: "kube-api-access-sj6j2") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "kube-api-access-sj6j2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:00:39.850376 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.850347    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:39.850463 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.850438    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:39.850737 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.850713    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-config-out" (OuterVolumeSpecName: "config-out") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:00:39.850927 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.850908    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:39.856135 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.856116    2567 scope.go:117] "RemoveContainer" containerID="288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"
Sep 18 20:00:39.862456 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.862432    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-web-config" (OuterVolumeSpecName: "web-config") pod "17177344-0e8b-4945-96f5-69259433ea2a" (UID: "17177344-0e8b-4945-96f5-69259433ea2a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:00:39.863443 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.863426    2567 scope.go:117] "RemoveContainer" containerID="8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"
Sep 18 20:00:39.870236 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.870218    2567 scope.go:117] "RemoveContainer" containerID="c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"
Sep 18 20:00:39.870492 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:39.870475    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": container with ID starting with c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456 not found: ID does not exist" containerID="c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"
Sep 18 20:00:39.870601 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.870498    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"} err="failed to get container status \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": rpc error: code = NotFound desc = could not find container \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": container with ID starting with c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456 not found: ID does not exist"
Sep 18 20:00:39.870601 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.870513    2567 scope.go:117] "RemoveContainer" containerID="53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"
Sep 18 20:00:39.870784 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:39.870765    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": container with ID starting with 53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5 not found: ID does not exist" containerID="53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"
Sep 18 20:00:39.870850 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.870790    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"} err="failed to get container status \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": rpc error: code = NotFound desc = could not find container \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": container with ID starting with 53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5 not found: ID does not exist"
Sep 18 20:00:39.870850 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.870807    2567 scope.go:117] "RemoveContainer" containerID="9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"
Sep 18 20:00:39.871065 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:39.871049    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": container with ID starting with 9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15 not found: ID does not exist" containerID="9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"
Sep 18 20:00:39.871120 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.871070    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"} err="failed to get container status \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": rpc error: code = NotFound desc = could not find container \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": container with ID starting with 9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15 not found: ID does not exist"
Sep 18 20:00:39.871120 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.871084    2567 scope.go:117] "RemoveContainer" containerID="aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"
Sep 18 20:00:39.871314 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:39.871297    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": container with ID starting with aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c not found: ID does not exist" containerID="aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"
Sep 18 20:00:39.871370 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.871320    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"} err="failed to get container status \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": rpc error: code = NotFound desc = could not find container \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": container with ID starting with aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c not found: ID does not exist"
Sep 18 20:00:39.871370 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.871335    2567 scope.go:117] "RemoveContainer" containerID="feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"
Sep 18 20:00:39.871581 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:39.871564    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": container with ID starting with feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02 not found: ID does not exist" containerID="feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"
Sep 18 20:00:39.871632 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.871586    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"} err="failed to get container status \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": rpc error: code = NotFound desc = could not find container \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": container with ID starting with feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02 not found: ID does not exist"
Sep 18 20:00:39.871632 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.871599    2567 scope.go:117] "RemoveContainer" containerID="288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"
Sep 18 20:00:39.871871 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:39.871854    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": container with ID starting with 288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff not found: ID does not exist" containerID="288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"
Sep 18 20:00:39.871930 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.871874    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"} err="failed to get container status \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": rpc error: code = NotFound desc = could not find container \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": container with ID starting with 288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff not found: ID does not exist"
Sep 18 20:00:39.871930 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.871886    2567 scope.go:117] "RemoveContainer" containerID="8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"
Sep 18 20:00:39.872123 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:00:39.872107    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": container with ID starting with 8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15 not found: ID does not exist" containerID="8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"
Sep 18 20:00:39.872173 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.872131    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"} err="failed to get container status \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": rpc error: code = NotFound desc = could not find container \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": container with ID starting with 8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15 not found: ID does not exist"
Sep 18 20:00:39.872173 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.872150    2567 scope.go:117] "RemoveContainer" containerID="c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"
Sep 18 20:00:39.872419 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.872387    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"} err="failed to get container status \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": rpc error: code = NotFound desc = could not find container \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": container with ID starting with c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456 not found: ID does not exist"
Sep 18 20:00:39.872419 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.872415    2567 scope.go:117] "RemoveContainer" containerID="53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"
Sep 18 20:00:39.872700 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.872669    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"} err="failed to get container status \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": rpc error: code = NotFound desc = could not find container \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": container with ID starting with 53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5 not found: ID does not exist"
Sep 18 20:00:39.872773 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.872701    2567 scope.go:117] "RemoveContainer" containerID="9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"
Sep 18 20:00:39.872963 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.872947    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"} err="failed to get container status \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": rpc error: code = NotFound desc = could not find container \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": container with ID starting with 9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15 not found: ID does not exist"
Sep 18 20:00:39.873019 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.872965    2567 scope.go:117] "RemoveContainer" containerID="aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"
Sep 18 20:00:39.873219 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.873198    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"} err="failed to get container status \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": rpc error: code = NotFound desc = could not find container \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": container with ID starting with aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c not found: ID does not exist"
Sep 18 20:00:39.873271 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.873222    2567 scope.go:117] "RemoveContainer" containerID="feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"
Sep 18 20:00:39.873495 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.873466    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"} err="failed to get container status \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": rpc error: code = NotFound desc = could not find container \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": container with ID starting with feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02 not found: ID does not exist"
Sep 18 20:00:39.873495 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.873490    2567 scope.go:117] "RemoveContainer" containerID="288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"
Sep 18 20:00:39.873804 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.873783    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"} err="failed to get container status \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": rpc error: code = NotFound desc = could not find container \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": container with ID starting with 288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff not found: ID does not exist"
Sep 18 20:00:39.873804 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.873802    2567 scope.go:117] "RemoveContainer" containerID="8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"
Sep 18 20:00:39.874054 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874035    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"} err="failed to get container status \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": rpc error: code = NotFound desc = could not find container \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": container with ID starting with 8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15 not found: ID does not exist"
Sep 18 20:00:39.874102 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874054    2567 scope.go:117] "RemoveContainer" containerID="c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"
Sep 18 20:00:39.874295 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874279    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"} err="failed to get container status \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": rpc error: code = NotFound desc = could not find container \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": container with ID starting with c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456 not found: ID does not exist"
Sep 18 20:00:39.874343 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874296    2567 scope.go:117] "RemoveContainer" containerID="53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"
Sep 18 20:00:39.874542 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874523    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"} err="failed to get container status \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": rpc error: code = NotFound desc = could not find container \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": container with ID starting with 53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5 not found: ID does not exist"
Sep 18 20:00:39.874542 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874541    2567 scope.go:117] "RemoveContainer" containerID="9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"
Sep 18 20:00:39.874758 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874744    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"} err="failed to get container status \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": rpc error: code = NotFound desc = could not find container \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": container with ID starting with 9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15 not found: ID does not exist"
Sep 18 20:00:39.874811 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874758    2567 scope.go:117] "RemoveContainer" containerID="aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"
Sep 18 20:00:39.874981 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874959    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"} err="failed to get container status \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": rpc error: code = NotFound desc = could not find container \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": container with ID starting with aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c not found: ID does not exist"
Sep 18 20:00:39.874981 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.874981    2567 scope.go:117] "RemoveContainer" containerID="feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"
Sep 18 20:00:39.875223 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.875204    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"} err="failed to get container status \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": rpc error: code = NotFound desc = could not find container \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": container with ID starting with feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02 not found: ID does not exist"
Sep 18 20:00:39.875278 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.875223    2567 scope.go:117] "RemoveContainer" containerID="288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"
Sep 18 20:00:39.875463 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.875445    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"} err="failed to get container status \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": rpc error: code = NotFound desc = could not find container \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": container with ID starting with 288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff not found: ID does not exist"
Sep 18 20:00:39.875533 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.875465    2567 scope.go:117] "RemoveContainer" containerID="8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"
Sep 18 20:00:39.875769 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.875677    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"} err="failed to get container status \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": rpc error: code = NotFound desc = could not find container \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": container with ID starting with 8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15 not found: ID does not exist"
Sep 18 20:00:39.875769 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.875715    2567 scope.go:117] "RemoveContainer" containerID="c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"
Sep 18 20:00:39.875981 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.875963    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"} err="failed to get container status \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": rpc error: code = NotFound desc = could not find container \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": container with ID starting with c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456 not found: ID does not exist"
Sep 18 20:00:39.876060 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.875982    2567 scope.go:117] "RemoveContainer" containerID="53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"
Sep 18 20:00:39.876270 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.876247    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"} err="failed to get container status \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": rpc error: code = NotFound desc = could not find container \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": container with ID starting with 53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5 not found: ID does not exist"
Sep 18 20:00:39.876270 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.876270    2567 scope.go:117] "RemoveContainer" containerID="9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"
Sep 18 20:00:39.876541 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.876513    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"} err="failed to get container status \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": rpc error: code = NotFound desc = could not find container \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": container with ID starting with 9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15 not found: ID does not exist"
Sep 18 20:00:39.876593 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.876543    2567 scope.go:117] "RemoveContainer" containerID="aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"
Sep 18 20:00:39.876842 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.876824    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"} err="failed to get container status \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": rpc error: code = NotFound desc = could not find container \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": container with ID starting with aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c not found: ID does not exist"
Sep 18 20:00:39.876890 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.876843    2567 scope.go:117] "RemoveContainer" containerID="feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"
Sep 18 20:00:39.877071 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.877053    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"} err="failed to get container status \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": rpc error: code = NotFound desc = could not find container \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": container with ID starting with feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02 not found: ID does not exist"
Sep 18 20:00:39.877130 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.877073    2567 scope.go:117] "RemoveContainer" containerID="288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"
Sep 18 20:00:39.877362 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.877344    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"} err="failed to get container status \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": rpc error: code = NotFound desc = could not find container \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": container with ID starting with 288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff not found: ID does not exist"
Sep 18 20:00:39.877415 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.877362    2567 scope.go:117] "RemoveContainer" containerID="8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"
Sep 18 20:00:39.877615 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.877596    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"} err="failed to get container status \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": rpc error: code = NotFound desc = could not find container \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": container with ID starting with 8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15 not found: ID does not exist"
Sep 18 20:00:39.877665 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.877616    2567 scope.go:117] "RemoveContainer" containerID="c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"
Sep 18 20:00:39.877885 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.877864    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"} err="failed to get container status \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": rpc error: code = NotFound desc = could not find container \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": container with ID starting with c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456 not found: ID does not exist"
Sep 18 20:00:39.877885 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.877885    2567 scope.go:117] "RemoveContainer" containerID="53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"
Sep 18 20:00:39.878152 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.878132    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"} err="failed to get container status \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": rpc error: code = NotFound desc = could not find container \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": container with ID starting with 53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5 not found: ID does not exist"
Sep 18 20:00:39.878152 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.878152    2567 scope.go:117] "RemoveContainer" containerID="9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"
Sep 18 20:00:39.878422 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.878405    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"} err="failed to get container status \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": rpc error: code = NotFound desc = could not find container \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": container with ID starting with 9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15 not found: ID does not exist"
Sep 18 20:00:39.878478 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.878422    2567 scope.go:117] "RemoveContainer" containerID="aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"
Sep 18 20:00:39.878653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.878630    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"} err="failed to get container status \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": rpc error: code = NotFound desc = could not find container \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": container with ID starting with aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c not found: ID does not exist"
Sep 18 20:00:39.878739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.878655    2567 scope.go:117] "RemoveContainer" containerID="feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"
Sep 18 20:00:39.880332 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.880293    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"} err="failed to get container status \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": rpc error: code = NotFound desc = could not find container \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": container with ID starting with feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02 not found: ID does not exist"
Sep 18 20:00:39.880332 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.880318    2567 scope.go:117] "RemoveContainer" containerID="288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"
Sep 18 20:00:39.880626 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.880594    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"} err="failed to get container status \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": rpc error: code = NotFound desc = could not find container \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": container with ID starting with 288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff not found: ID does not exist"
Sep 18 20:00:39.880730 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.880628    2567 scope.go:117] "RemoveContainer" containerID="8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"
Sep 18 20:00:39.880929 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.880906    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"} err="failed to get container status \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": rpc error: code = NotFound desc = could not find container \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": container with ID starting with 8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15 not found: ID does not exist"
Sep 18 20:00:39.881004 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.880930    2567 scope.go:117] "RemoveContainer" containerID="c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"
Sep 18 20:00:39.881182 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.881158    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456"} err="failed to get container status \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": rpc error: code = NotFound desc = could not find container \"c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456\": container with ID starting with c74b94d182c8c609b2998f4d0cc65776c190782147d1c2170474c3756182a456 not found: ID does not exist"
Sep 18 20:00:39.881264 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.881182    2567 scope.go:117] "RemoveContainer" containerID="53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"
Sep 18 20:00:39.881431 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.881411    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5"} err="failed to get container status \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": rpc error: code = NotFound desc = could not find container \"53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5\": container with ID starting with 53b190fca8ea6f8f0c31e147d5695554cf754e3915d9992bdf1e943c371782c5 not found: ID does not exist"
Sep 18 20:00:39.881506 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.881433    2567 scope.go:117] "RemoveContainer" containerID="9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"
Sep 18 20:00:39.881703 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.881664    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15"} err="failed to get container status \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": rpc error: code = NotFound desc = could not find container \"9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15\": container with ID starting with 9ad0281d7e75428db3fad430bbbb743b8664852bd6accf11b6ff87d1db34cd15 not found: ID does not exist"
Sep 18 20:00:39.881767 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.881714    2567 scope.go:117] "RemoveContainer" containerID="aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"
Sep 18 20:00:39.881968 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.881951    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c"} err="failed to get container status \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": rpc error: code = NotFound desc = could not find container \"aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c\": container with ID starting with aa67bb62b6ba202588fe2db4f4901e6379c91e70d7d5ad005b7a9acc2e55f97c not found: ID does not exist"
Sep 18 20:00:39.881968 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.881968    2567 scope.go:117] "RemoveContainer" containerID="feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"
Sep 18 20:00:39.882205 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.882186    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02"} err="failed to get container status \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": rpc error: code = NotFound desc = could not find container \"feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02\": container with ID starting with feef2b0c9932ac178b155a85650b4409eb4a35eb1a6f27ce4209da9cf0a3cc02 not found: ID does not exist"
Sep 18 20:00:39.882247 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.882206    2567 scope.go:117] "RemoveContainer" containerID="288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"
Sep 18 20:00:39.882828 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.882810    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff"} err="failed to get container status \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": rpc error: code = NotFound desc = could not find container \"288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff\": container with ID starting with 288d34e557e6b5e4d08fafc745eb7dc5bf5634412f908622c4551b662adf6cff not found: ID does not exist"
Sep 18 20:00:39.882828 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.882827    2567 scope.go:117] "RemoveContainer" containerID="8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"
Sep 18 20:00:39.883086 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.883068    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15"} err="failed to get container status \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": rpc error: code = NotFound desc = could not find container \"8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15\": container with ID starting with 8b9a6bea00e34dabaa738a476f0a51647d811ff3be0a3035c7cb168c3fd6cb15 not found: ID does not exist"
Sep 18 20:00:39.945447 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945427    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945447 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945446    2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-kube-rbac-proxy\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945458    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945467    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945475    2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945483    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-metrics-client-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945491    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945499    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-tls-assets\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945507    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-config-out\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945514    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-db\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945521    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945529    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945536    2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-grpc-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945857 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945544    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945857 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945552    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945857 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945560    2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-secret-metrics-client-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945857 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945567    2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-config\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945857 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945574    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/17177344-0e8b-4945-96f5-69259433ea2a-web-config\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945857 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945582    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/17177344-0e8b-4945-96f5-69259433ea2a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:39.945857 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:39.945589    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sj6j2\" (UniqueName: \"kubernetes.io/projected/17177344-0e8b-4945-96f5-69259433ea2a-kube-api-access-sj6j2\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:00:40.138681 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.138647    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 20:00:40.145573 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.145548    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 20:00:40.178735 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.178715    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 20:00:40.179002 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.178991    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="thanos-sidecar"
Sep 18 20:00:40.179048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179004    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="thanos-sidecar"
Sep 18 20:00:40.179048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179014    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="init-config-reloader"
Sep 18 20:00:40.179048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179019    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="init-config-reloader"
Sep 18 20:00:40.179048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179029    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="prometheus"
Sep 18 20:00:40.179048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179035    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="prometheus"
Sep 18 20:00:40.179048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179041    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy-web"
Sep 18 20:00:40.179048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179046    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy-web"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179056    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179061    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179067    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy-thanos"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179071    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy-thanos"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179077    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="config-reloader"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179082    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="config-reloader"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179129    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179136    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="config-reloader"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179142    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy-web"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179148    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="prometheus"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179154    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="thanos-sidecar"
Sep 18 20:00:40.179234 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.179159    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="17177344-0e8b-4945-96f5-69259433ea2a" containerName="kube-rbac-proxy-thanos"
Sep 18 20:00:40.183994 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.183979    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.186385 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.186361    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 18 20:00:40.186484 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.186403    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 18 20:00:40.186484 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.186414    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 18 20:00:40.186594 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.186574    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 18 20:00:40.186775 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.186761    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 18 20:00:40.186836 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.186788    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 18 20:00:40.186988 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.186973    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 18 20:00:40.187131 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.187118    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 18 20:00:40.187180 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.187142    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 18 20:00:40.187537 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.187521    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 18 20:00:40.187717 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.187704    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 18 20:00:40.187795 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.187767    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 18 20:00:40.187927 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.187909    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-aps6ardtfvk6n\""
Sep 18 20:00:40.190002 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.189979    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bzjsb\""
Sep 18 20:00:40.190908 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.190891    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 18 20:00:40.195121 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.195104    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 18 20:00:40.204349 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.204329    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 20:00:40.248670 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248650    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.248758 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248681    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.248758 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248730    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0ed0f345-b86e-42f5-a058-a6fc4398e901-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.248758 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248745    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.248875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248766    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.248875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248782    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0ed0f345-b86e-42f5-a058-a6fc4398e901-config-out\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.248875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248796    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.248875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248841    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-config\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249061 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249061 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248928    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249061 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248962    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249061 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.248993    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249061 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.249030    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249243 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.249067    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nmwv\" (UniqueName: \"kubernetes.io/projected/0ed0f345-b86e-42f5-a058-a6fc4398e901-kube-api-access-8nmwv\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249243 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.249108    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249243 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.249141    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-web-config\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249243 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.249199    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249243 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.249241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249381 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.249260    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.249381 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.249275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.350584 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.350560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.350679 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.350587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.350679 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.350605    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.350679 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.350620    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.350679 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.350635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.350679 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.350665    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.350996 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.350882    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0ed0f345-b86e-42f5-a058-a6fc4398e901-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.350996 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.350929    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.350996 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.350971    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351150 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0ed0f345-b86e-42f5-a058-a6fc4398e901-config-out\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351150 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351024    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351150 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351049    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-config\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351150 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351150 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351150 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351134    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351427 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351159    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351427 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351191    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351427 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351226    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nmwv\" (UniqueName: \"kubernetes.io/projected/0ed0f345-b86e-42f5-a058-a6fc4398e901-kube-api-access-8nmwv\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351427 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351427 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351302    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-web-config\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.351427 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351378    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.352079 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351839    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.352079 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.351909    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.352213 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.352102    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.352598 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.352569    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.353165 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.353137    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.353282 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.353257    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.353755 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.353711    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.354477 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.354450    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.354598 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.354567    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.355053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.355030    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-config\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.355152 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.355036    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.355321 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.355298    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0ed0f345-b86e-42f5-a058-a6fc4398e901-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.355551 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.355531    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0ed0f345-b86e-42f5-a058-a6fc4398e901-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.355780 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.355762    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-web-config\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.355960 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.355942    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.356280 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.356262    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.356531 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.356513    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0ed0f345-b86e-42f5-a058-a6fc4398e901-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.356850 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.356827    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0ed0f345-b86e-42f5-a058-a6fc4398e901-config-out\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.360961 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.360945    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nmwv\" (UniqueName: \"kubernetes.io/projected/0ed0f345-b86e-42f5-a058-a6fc4398e901-kube-api-access-8nmwv\") pod \"prometheus-k8s-0\" (UID: \"0ed0f345-b86e-42f5-a058-a6fc4398e901\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.493155 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.493093    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:00:40.772661 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.772602    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 20:00:40.775002 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:00:40.774971    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed0f345_b86e_42f5_a058_a6fc4398e901.slice/crio-0cc181293f16d339f475d9fdf4c087549053954ea241e5b55b36ae9b23b94bf6 WatchSource:0}: Error finding container 0cc181293f16d339f475d9fdf4c087549053954ea241e5b55b36ae9b23b94bf6: Status 404 returned error can't find the container with id 0cc181293f16d339f475d9fdf4c087549053954ea241e5b55b36ae9b23b94bf6
Sep 18 20:00:40.820315 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:40.820288    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed0f345-b86e-42f5-a058-a6fc4398e901","Type":"ContainerStarted","Data":"0cc181293f16d339f475d9fdf4c087549053954ea241e5b55b36ae9b23b94bf6"}
Sep 18 20:00:41.698599 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:41.698564    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17177344-0e8b-4945-96f5-69259433ea2a" path="/var/lib/kubelet/pods/17177344-0e8b-4945-96f5-69259433ea2a/volumes"
Sep 18 20:00:41.825097 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:41.825035    2567 generic.go:356] "Generic (PLEG): container finished" podID="0ed0f345-b86e-42f5-a058-a6fc4398e901" containerID="695386440c8e4e046fa4edb07c8ca420a07f4b3e1d560235d5355a112217d97e" exitCode=0
Sep 18 20:00:41.825097 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:41.825065    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed0f345-b86e-42f5-a058-a6fc4398e901","Type":"ContainerDied","Data":"695386440c8e4e046fa4edb07c8ca420a07f4b3e1d560235d5355a112217d97e"}
Sep 18 20:00:42.833673 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:42.833637    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed0f345-b86e-42f5-a058-a6fc4398e901","Type":"ContainerStarted","Data":"ad059c2fd55fd52434977ccf0dc43478e98dab586d5fa8f774b47cd7cd8a5983"}
Sep 18 20:00:42.834074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:42.833721    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed0f345-b86e-42f5-a058-a6fc4398e901","Type":"ContainerStarted","Data":"62fd52fd08e3a556c2f27a47a6ab3ba3ef427213d753919d6be824354c6d6c31"}
Sep 18 20:00:42.834074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:42.833753    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed0f345-b86e-42f5-a058-a6fc4398e901","Type":"ContainerStarted","Data":"7fabd17519b00956468917fbf71077805d34a987fd5b323f59ddb32f10bb66d9"}
Sep 18 20:00:42.834074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:42.833780    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed0f345-b86e-42f5-a058-a6fc4398e901","Type":"ContainerStarted","Data":"fd1935fd9dd4b0cb539f1f76a0fdb8db8342c233ef2250ffce5abcfda46b58db"}
Sep 18 20:00:42.834074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:42.833809    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed0f345-b86e-42f5-a058-a6fc4398e901","Type":"ContainerStarted","Data":"3ebb1affae2fc21282bcea79a76bcf44dc1768d13431d3c6de65f757a6f6344e"}
Sep 18 20:00:42.834074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:42.833833    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0ed0f345-b86e-42f5-a058-a6fc4398e901","Type":"ContainerStarted","Data":"1f36b4c066c2988b8d8905cf0aac836f0f4e848611d87b78b74e97eca87fe712"}
Sep 18 20:00:42.864787 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:42.864732    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.846820668 podStartE2EDuration="2.864713236s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.892568ms" isStatefulPod=false podCreationTimestamp="2026-09-18 20:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:00:42.862947181 +0000 UTC m=+223.423530637" watchObservedRunningTime="2026-09-18 20:00:42.864713236 +0000 UTC m=+223.425296694"
Sep 18 20:00:45.493543 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:00:45.493501    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:01:40.493785 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:01:40.493740    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:01:40.509183 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:01:40.509160    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:01:41.084627 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:01:41.084602    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:02:01.599630 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:02:01.599605    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:02:01.602020 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:02:01.601999    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:02:01.610714 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:02:01.610680    2567 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 20:03:50.205668 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.205631    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"]
Sep 18 20:03:50.208927 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.208911    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"
Sep 18 20:03:50.212209 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.212185    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 20:03:50.213104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.213078    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-l7fst\""
Sep 18 20:03:50.213104 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.213078    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:03:50.219903 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.219879    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"]
Sep 18 20:03:50.303990 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.303958    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc2cd8f0-bf11-40d2-a9d3-d8c48411944e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zjzxj\" (UID: \"dc2cd8f0-bf11-40d2-a9d3-d8c48411944e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"
Sep 18 20:03:50.304144 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.303996    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7fgf\" (UniqueName: \"kubernetes.io/projected/dc2cd8f0-bf11-40d2-a9d3-d8c48411944e-kube-api-access-s7fgf\") pod \"openshift-lws-operator-fd8ccff4c-zjzxj\" (UID: \"dc2cd8f0-bf11-40d2-a9d3-d8c48411944e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"
Sep 18 20:03:50.404934 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.404901    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc2cd8f0-bf11-40d2-a9d3-d8c48411944e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zjzxj\" (UID: \"dc2cd8f0-bf11-40d2-a9d3-d8c48411944e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"
Sep 18 20:03:50.405058 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.404939    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7fgf\" (UniqueName: \"kubernetes.io/projected/dc2cd8f0-bf11-40d2-a9d3-d8c48411944e-kube-api-access-s7fgf\") pod \"openshift-lws-operator-fd8ccff4c-zjzxj\" (UID: \"dc2cd8f0-bf11-40d2-a9d3-d8c48411944e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"
Sep 18 20:03:50.405287 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.405267    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc2cd8f0-bf11-40d2-a9d3-d8c48411944e-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zjzxj\" (UID: \"dc2cd8f0-bf11-40d2-a9d3-d8c48411944e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"
Sep 18 20:03:50.413606 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.413578    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7fgf\" (UniqueName: \"kubernetes.io/projected/dc2cd8f0-bf11-40d2-a9d3-d8c48411944e-kube-api-access-s7fgf\") pod \"openshift-lws-operator-fd8ccff4c-zjzxj\" (UID: \"dc2cd8f0-bf11-40d2-a9d3-d8c48411944e\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"
Sep 18 20:03:50.528741 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.528655    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"
Sep 18 20:03:50.796219 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.796191    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj"]
Sep 18 20:03:50.798505 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:03:50.798477    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc2cd8f0_bf11_40d2_a9d3_d8c48411944e.slice/crio-2bdfd32694eb678446a7a546906a31ae1d5a6cf02ca14281e141cd72a2dd95ae WatchSource:0}: Error finding container 2bdfd32694eb678446a7a546906a31ae1d5a6cf02ca14281e141cd72a2dd95ae: Status 404 returned error can't find the container with id 2bdfd32694eb678446a7a546906a31ae1d5a6cf02ca14281e141cd72a2dd95ae
Sep 18 20:03:50.800046 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:50.800027    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:03:51.596719 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:51.596669    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj" event={"ID":"dc2cd8f0-bf11-40d2-a9d3-d8c48411944e","Type":"ContainerStarted","Data":"2bdfd32694eb678446a7a546906a31ae1d5a6cf02ca14281e141cd72a2dd95ae"}
Sep 18 20:03:54.610961 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:54.610925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj" event={"ID":"dc2cd8f0-bf11-40d2-a9d3-d8c48411944e","Type":"ContainerStarted","Data":"acbed01a1ad096642b654650b81100573612d01fe472581380f672fb00a738f3"}
Sep 18 20:03:54.629553 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:03:54.629506    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zjzxj" podStartSLOduration=1.832507441 podStartE2EDuration="4.629490764s" totalImagesPullingTime="2.796983323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:03:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:03:54.627554095 +0000 UTC m=+415.188137551" watchObservedRunningTime="2026-09-18 20:03:54.629490764 +0000 UTC m=+415.190074218"
Sep 18 20:04:01.666084 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.666054    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"]
Sep 18 20:04:01.669398 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.669382    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.672592 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.672561    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 18 20:04:01.672717 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.672569    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-jcn2q\""
Sep 18 20:04:01.672717 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.672569    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 18 20:04:01.672717 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.672662    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 18 20:04:01.683356 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.683338    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"]
Sep 18 20:04:01.796401 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.796370    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/216ce354-df51-4524-9aac-0258d59c7948-manager-config\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.796559 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.796445    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9dpp\" (UniqueName: \"kubernetes.io/projected/216ce354-df51-4524-9aac-0258d59c7948-kube-api-access-g9dpp\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.796559 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.796491    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/216ce354-df51-4524-9aac-0258d59c7948-cert\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.796644 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.796569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/216ce354-df51-4524-9aac-0258d59c7948-metrics-cert\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.897653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.897624    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9dpp\" (UniqueName: \"kubernetes.io/projected/216ce354-df51-4524-9aac-0258d59c7948-kube-api-access-g9dpp\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.897812 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.897659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/216ce354-df51-4524-9aac-0258d59c7948-cert\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.897812 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.897715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/216ce354-df51-4524-9aac-0258d59c7948-metrics-cert\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.897812 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.897734    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/216ce354-df51-4524-9aac-0258d59c7948-manager-config\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.898333 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.898312    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/216ce354-df51-4524-9aac-0258d59c7948-manager-config\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.900112 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.900094    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/216ce354-df51-4524-9aac-0258d59c7948-metrics-cert\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.900194 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.900139    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/216ce354-df51-4524-9aac-0258d59c7948-cert\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.913459 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.913429    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9dpp\" (UniqueName: \"kubernetes.io/projected/216ce354-df51-4524-9aac-0258d59c7948-kube-api-access-g9dpp\") pod \"lws-controller-manager-5848b5686c-hmtqx\" (UID: \"216ce354-df51-4524-9aac-0258d59c7948\") " pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:01.977984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:01.977929    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:02.279225 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:02.279060    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"]
Sep 18 20:04:02.281425 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:04:02.281400    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216ce354_df51_4524_9aac_0258d59c7948.slice/crio-7c1c1db44826224c642077afddb5bc09f303d9d440a1bba054bc670abddf3fc7 WatchSource:0}: Error finding container 7c1c1db44826224c642077afddb5bc09f303d9d440a1bba054bc670abddf3fc7: Status 404 returned error can't find the container with id 7c1c1db44826224c642077afddb5bc09f303d9d440a1bba054bc670abddf3fc7
Sep 18 20:04:02.644092 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:02.644058    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx" event={"ID":"216ce354-df51-4524-9aac-0258d59c7948","Type":"ContainerStarted","Data":"7c1c1db44826224c642077afddb5bc09f303d9d440a1bba054bc670abddf3fc7"}
Sep 18 20:04:04.654744 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:04.654709    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx" event={"ID":"216ce354-df51-4524-9aac-0258d59c7948","Type":"ContainerStarted","Data":"c814295a6709d3e1d26499e87dd971e078328c1e860b4cfa8f5cad8e20b29c2b"}
Sep 18 20:04:04.655092 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:04.654856    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:04.672448 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:04.672326    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx" podStartSLOduration=2.202362662 podStartE2EDuration="3.67230864s" totalImagesPullingTime="1.469945978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:04:04.671826687 +0000 UTC m=+425.232410138" watchObservedRunningTime="2026-09-18 20:04:04.67230864 +0000 UTC m=+425.232892100"
Sep 18 20:04:05.159696 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.159651    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59d7cd6bb6-rspnq"]
Sep 18 20:04:05.163130 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.163110    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.166235 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.166219    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 20:04:05.166711 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.166695    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 20:04:05.169669 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.169605    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 20:04:05.170199 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.169927    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 20:04:05.170361 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.170343    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 20:04:05.170837 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.170815    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 20:04:05.171265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.171247    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4v52r\""
Sep 18 20:04:05.171359 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.171260    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 20:04:05.175324 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.175306    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 20:04:05.178765 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.178747    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59d7cd6bb6-rspnq"]
Sep 18 20:04:05.229189 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.229156    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-service-ca\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.229364 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.229204    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-console-oauth-config\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.229364 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.229232    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-trusted-ca-bundle\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.229364 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.229348    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-console-serving-cert\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.229495 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.229382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-console-config\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.229495 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.229419    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-oauth-serving-cert\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.229574 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.229489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v26wf\" (UniqueName: \"kubernetes.io/projected/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-kube-api-access-v26wf\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.330892 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.330858    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-oauth-serving-cert\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.331053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.330905    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v26wf\" (UniqueName: \"kubernetes.io/projected/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-kube-api-access-v26wf\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.331053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.330935    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-service-ca\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.331053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.330952    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-console-oauth-config\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.331053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.330970    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-trusted-ca-bundle\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.331053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.331001    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-console-serving-cert\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.331053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.331022    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-console-config\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.331747 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.331719    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-oauth-serving-cert\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.331888 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.331786    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-console-config\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.331888 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.331858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-trusted-ca-bundle\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.332016 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.331943    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-service-ca\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.333339 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.333320    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-console-oauth-config\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.333481 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.333453    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-console-serving-cert\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.349803 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.349782    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v26wf\" (UniqueName: \"kubernetes.io/projected/2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4-kube-api-access-v26wf\") pod \"console-59d7cd6bb6-rspnq\" (UID: \"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4\") " pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.473663 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.473586    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:05.753705 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:05.753573    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59d7cd6bb6-rspnq"]
Sep 18 20:04:05.755887 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:04:05.755862    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2561e8ac_3e9f_49b0_820f_ad7fb0eaa1f4.slice/crio-014bcfb0e85d88946c34cf19292071014110c1d399bd98609cd54d89dff60315 WatchSource:0}: Error finding container 014bcfb0e85d88946c34cf19292071014110c1d399bd98609cd54d89dff60315: Status 404 returned error can't find the container with id 014bcfb0e85d88946c34cf19292071014110c1d399bd98609cd54d89dff60315
Sep 18 20:04:06.665275 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:06.665243    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d7cd6bb6-rspnq" event={"ID":"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4","Type":"ContainerStarted","Data":"43332dd3ea936c9aa31d37d653a079817b008b8d3d782abbde7a0fc6950b5079"}
Sep 18 20:04:06.665445 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:06.665286    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d7cd6bb6-rspnq" event={"ID":"2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4","Type":"ContainerStarted","Data":"014bcfb0e85d88946c34cf19292071014110c1d399bd98609cd54d89dff60315"}
Sep 18 20:04:06.685197 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:06.685139    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59d7cd6bb6-rspnq" podStartSLOduration=1.685120694 podStartE2EDuration="1.685120694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:04:06.683630571 +0000 UTC m=+427.244214029" watchObservedRunningTime="2026-09-18 20:04:06.685120694 +0000 UTC m=+427.245704152"
Sep 18 20:04:14.072215 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.072119    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"]
Sep 18 20:04:14.076649 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.076618    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.080538 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.080511    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 20:04:14.080892 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.080869    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 20:04:14.081340 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.081320    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 20:04:14.081443 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.081338    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cphdw\""
Sep 18 20:04:14.081633 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.081614    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 20:04:14.117173 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.117140    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"]
Sep 18 20:04:14.209755 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.209721    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f7909318-b051-4381-81e1-ba519a25722e-manifests\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.209948 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.209781    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f7909318-b051-4381-81e1-ba519a25722e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.209948 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.209848    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7909318-b051-4381-81e1-ba519a25722e-tmp\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.209948 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.209922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7909318-b051-4381-81e1-ba519a25722e-webhook-cert\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.210107 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.209996    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65j5d\" (UniqueName: \"kubernetes.io/projected/f7909318-b051-4381-81e1-ba519a25722e-kube-api-access-65j5d\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.311272 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.311239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f7909318-b051-4381-81e1-ba519a25722e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.311272 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.311275    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7909318-b051-4381-81e1-ba519a25722e-tmp\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.311496 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.311316    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7909318-b051-4381-81e1-ba519a25722e-webhook-cert\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.311496 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.311377    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65j5d\" (UniqueName: \"kubernetes.io/projected/f7909318-b051-4381-81e1-ba519a25722e-kube-api-access-65j5d\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.311496 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.311441    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f7909318-b051-4381-81e1-ba519a25722e-manifests\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.311825 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.311802    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f7909318-b051-4381-81e1-ba519a25722e-manifests\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.313497 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.313476    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f7909318-b051-4381-81e1-ba519a25722e-tmp\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.313766 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.313748    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f7909318-b051-4381-81e1-ba519a25722e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.313847 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.313830    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f7909318-b051-4381-81e1-ba519a25722e-webhook-cert\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.325909 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.325845    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65j5d\" (UniqueName: \"kubernetes.io/projected/f7909318-b051-4381-81e1-ba519a25722e-kube-api-access-65j5d\") pod \"opendatahub-operator-controller-manager-f76f79b89-gdn8l\" (UID: \"f7909318-b051-4381-81e1-ba519a25722e\") " pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.388821 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.388792    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:14.675850 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.675826    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"]
Sep 18 20:04:14.677655 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:04:14.677628    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7909318_b051_4381_81e1_ba519a25722e.slice/crio-6e71ae191cfc3b3cbe10dd9760f2a0a7fc0fd7975fef01f57256e2cadf2a9bbb WatchSource:0}: Error finding container 6e71ae191cfc3b3cbe10dd9760f2a0a7fc0fd7975fef01f57256e2cadf2a9bbb: Status 404 returned error can't find the container with id 6e71ae191cfc3b3cbe10dd9760f2a0a7fc0fd7975fef01f57256e2cadf2a9bbb
Sep 18 20:04:14.702297 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:14.702273    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l" event={"ID":"f7909318-b051-4381-81e1-ba519a25722e","Type":"ContainerStarted","Data":"6e71ae191cfc3b3cbe10dd9760f2a0a7fc0fd7975fef01f57256e2cadf2a9bbb"}
Sep 18 20:04:15.474052 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:15.473971    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:15.474052 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:15.474043    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:15.480598 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:15.480572    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:15.662519 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:15.662294    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5848b5686c-hmtqx"
Sep 18 20:04:15.715026 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:15.715002    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59d7cd6bb6-rspnq"
Sep 18 20:04:18.727343 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:18.727305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l" event={"ID":"f7909318-b051-4381-81e1-ba519a25722e","Type":"ContainerStarted","Data":"21034a440bb9f154d1b2ddbcad30848bcdc484c232fcd5426954827c2e8e35f8"}
Sep 18 20:04:19.732636 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:19.732606    2567 generic.go:356] "Generic (PLEG): container finished" podID="f7909318-b051-4381-81e1-ba519a25722e" containerID="21034a440bb9f154d1b2ddbcad30848bcdc484c232fcd5426954827c2e8e35f8" exitCode=0
Sep 18 20:04:19.733133 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:19.732648    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l" event={"ID":"f7909318-b051-4381-81e1-ba519a25722e","Type":"ContainerDied","Data":"21034a440bb9f154d1b2ddbcad30848bcdc484c232fcd5426954827c2e8e35f8"}
Sep 18 20:04:20.739072 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:20.739036    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l" event={"ID":"f7909318-b051-4381-81e1-ba519a25722e","Type":"ContainerStarted","Data":"3634f0e16a789357fd8df7a2d07964b546fbdf0bbd520c3a28c21d26dbc76936"}
Sep 18 20:04:20.739466 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:20.739178    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:20.780928 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:20.780884    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l" podStartSLOduration=2.29756572 podStartE2EDuration="6.780869129s" totalImagesPullingTime="3.428940378s" totalInitContainerRuntime="1.054363031s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:04:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:04:20.778765318 +0000 UTC m=+441.339348777" watchObservedRunningTime="2026-09-18 20:04:20.780869129 +0000 UTC m=+441.341452586"
Sep 18 20:04:31.745654 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:31.745624    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-f76f79b89-gdn8l"
Sep 18 20:04:34.646541 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.646509    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"]
Sep 18 20:04:34.653556 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.653526    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.664980 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.664963    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 20:04:34.680110 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.680090    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 20:04:34.681057 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.681042    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 20:04:34.684441 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.684423    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7plg9\""
Sep 18 20:04:34.693308 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.693286    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a313b08-2ac0-4200-ab53-f359a06cb932-tmp\") pod \"kube-auth-proxy-6dc98cf6f8-zx5xc\" (UID: \"1a313b08-2ac0-4200-ab53-f359a06cb932\") " pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.693378 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.693335    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgftd\" (UniqueName: \"kubernetes.io/projected/1a313b08-2ac0-4200-ab53-f359a06cb932-kube-api-access-sgftd\") pod \"kube-auth-proxy-6dc98cf6f8-zx5xc\" (UID: \"1a313b08-2ac0-4200-ab53-f359a06cb932\") " pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.693431 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.693415    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a313b08-2ac0-4200-ab53-f359a06cb932-tls-certs\") pod \"kube-auth-proxy-6dc98cf6f8-zx5xc\" (UID: \"1a313b08-2ac0-4200-ab53-f359a06cb932\") " pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.708987 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.708966    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 20:04:34.767939 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.767916    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"]
Sep 18 20:04:34.794675 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.794647    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a313b08-2ac0-4200-ab53-f359a06cb932-tmp\") pod \"kube-auth-proxy-6dc98cf6f8-zx5xc\" (UID: \"1a313b08-2ac0-4200-ab53-f359a06cb932\") " pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.794783 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.794730    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgftd\" (UniqueName: \"kubernetes.io/projected/1a313b08-2ac0-4200-ab53-f359a06cb932-kube-api-access-sgftd\") pod \"kube-auth-proxy-6dc98cf6f8-zx5xc\" (UID: \"1a313b08-2ac0-4200-ab53-f359a06cb932\") " pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.794783 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.794771    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a313b08-2ac0-4200-ab53-f359a06cb932-tls-certs\") pod \"kube-auth-proxy-6dc98cf6f8-zx5xc\" (UID: \"1a313b08-2ac0-4200-ab53-f359a06cb932\") " pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.796942 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.796912    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a313b08-2ac0-4200-ab53-f359a06cb932-tmp\") pod \"kube-auth-proxy-6dc98cf6f8-zx5xc\" (UID: \"1a313b08-2ac0-4200-ab53-f359a06cb932\") " pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.797096 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.797078    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a313b08-2ac0-4200-ab53-f359a06cb932-tls-certs\") pod \"kube-auth-proxy-6dc98cf6f8-zx5xc\" (UID: \"1a313b08-2ac0-4200-ab53-f359a06cb932\") " pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.843413 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.843379    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgftd\" (UniqueName: \"kubernetes.io/projected/1a313b08-2ac0-4200-ab53-f359a06cb932-kube-api-access-sgftd\") pod \"kube-auth-proxy-6dc98cf6f8-zx5xc\" (UID: \"1a313b08-2ac0-4200-ab53-f359a06cb932\") " pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:34.967611 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:34.967517    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"
Sep 18 20:04:35.309170 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:35.309108    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc"]
Sep 18 20:04:35.311898 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:04:35.311872    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a313b08_2ac0_4200_ab53_f359a06cb932.slice/crio-b8bd6442ad9cb9ffc827015d12680626457b1129c30e20d231177e1f23c2ba21 WatchSource:0}: Error finding container b8bd6442ad9cb9ffc827015d12680626457b1129c30e20d231177e1f23c2ba21: Status 404 returned error can't find the container with id b8bd6442ad9cb9ffc827015d12680626457b1129c30e20d231177e1f23c2ba21
Sep 18 20:04:35.807158 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:35.807126    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc" event={"ID":"1a313b08-2ac0-4200-ab53-f359a06cb932","Type":"ContainerStarted","Data":"b8bd6442ad9cb9ffc827015d12680626457b1129c30e20d231177e1f23c2ba21"}
Sep 18 20:04:38.824423 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:38.824380    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc" event={"ID":"1a313b08-2ac0-4200-ab53-f359a06cb932","Type":"ContainerStarted","Data":"067816d32286ec2e27eb387fc6b5327dd4994ba37b4bbe8ff1bb7f37c504d70d"}
Sep 18 20:04:38.844239 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:04:38.844194    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6dc98cf6f8-zx5xc" podStartSLOduration=2.141599508 podStartE2EDuration="4.844177451s" totalImagesPullingTime="2.702577943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:04:38.841845139 +0000 UTC m=+459.402428607" watchObservedRunningTime="2026-09-18 20:04:38.844177451 +0000 UTC m=+459.404760908"
Sep 18 20:07:01.643575 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:01.643542    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:07:01.644128 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:01.643994    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:07:04.368842 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.368811    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mpjrl"]
Sep 18 20:07:04.371274 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.371257    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:04.375775 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.375755    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 20:07:04.375886 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.375792    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 20:07:04.376327 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.376306    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 20:07:04.376434 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.376310    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4qclr\""
Sep 18 20:07:04.383909 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.383888    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mpjrl"]
Sep 18 20:07:04.434284 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.434247    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd682\" (UniqueName: \"kubernetes.io/projected/043fcad7-4ad5-4ec3-bbae-680f7267db1e-kube-api-access-pd682\") pod \"limitador-limitador-8588749977-mpjrl\" (UID: \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\") " pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:04.434452 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.434383    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/043fcad7-4ad5-4ec3-bbae-680f7267db1e-config-file\") pod \"limitador-limitador-8588749977-mpjrl\" (UID: \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\") " pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:04.444973 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.444943    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mpjrl"]
Sep 18 20:07:04.535811 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.535782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/043fcad7-4ad5-4ec3-bbae-680f7267db1e-config-file\") pod \"limitador-limitador-8588749977-mpjrl\" (UID: \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\") " pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:04.535973 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.535880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd682\" (UniqueName: \"kubernetes.io/projected/043fcad7-4ad5-4ec3-bbae-680f7267db1e-kube-api-access-pd682\") pod \"limitador-limitador-8588749977-mpjrl\" (UID: \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\") " pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:04.536410 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.536393    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/043fcad7-4ad5-4ec3-bbae-680f7267db1e-config-file\") pod \"limitador-limitador-8588749977-mpjrl\" (UID: \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\") " pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:04.545870 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.545838    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd682\" (UniqueName: \"kubernetes.io/projected/043fcad7-4ad5-4ec3-bbae-680f7267db1e-kube-api-access-pd682\") pod \"limitador-limitador-8588749977-mpjrl\" (UID: \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\") " pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:04.682067 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.681989    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:04.990971 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:04.990898    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mpjrl"]
Sep 18 20:07:04.994053 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:07:04.994029    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod043fcad7_4ad5_4ec3_bbae_680f7267db1e.slice/crio-f603cb379601eb0b9c171ae558b38cd6b9f68a930220e4bbd98e567a0346d7ca WatchSource:0}: Error finding container f603cb379601eb0b9c171ae558b38cd6b9f68a930220e4bbd98e567a0346d7ca: Status 404 returned error can't find the container with id f603cb379601eb0b9c171ae558b38cd6b9f68a930220e4bbd98e567a0346d7ca
Sep 18 20:07:05.506830 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:05.506799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl" event={"ID":"043fcad7-4ad5-4ec3-bbae-680f7267db1e","Type":"ContainerStarted","Data":"f603cb379601eb0b9c171ae558b38cd6b9f68a930220e4bbd98e567a0346d7ca"}
Sep 18 20:07:05.531000 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:05.530968    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-c8w9r"]
Sep 18 20:07:05.533632 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:05.533614    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-c8w9r"
Sep 18 20:07:05.536484 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:05.536468    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nhv9b\""
Sep 18 20:07:05.551540 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:05.551513    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-c8w9r"]
Sep 18 20:07:05.646527 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:05.646494    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tkpx\" (UniqueName: \"kubernetes.io/projected/3aedf792-6f12-4507-9585-1c0b1d74ccfa-kube-api-access-5tkpx\") pod \"authorino-59c974bd48-c8w9r\" (UID: \"3aedf792-6f12-4507-9585-1c0b1d74ccfa\") " pod="kuadrant-system/authorino-59c974bd48-c8w9r"
Sep 18 20:07:05.748070 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:05.747681    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tkpx\" (UniqueName: \"kubernetes.io/projected/3aedf792-6f12-4507-9585-1c0b1d74ccfa-kube-api-access-5tkpx\") pod \"authorino-59c974bd48-c8w9r\" (UID: \"3aedf792-6f12-4507-9585-1c0b1d74ccfa\") " pod="kuadrant-system/authorino-59c974bd48-c8w9r"
Sep 18 20:07:05.764000 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:05.763935    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tkpx\" (UniqueName: \"kubernetes.io/projected/3aedf792-6f12-4507-9585-1c0b1d74ccfa-kube-api-access-5tkpx\") pod \"authorino-59c974bd48-c8w9r\" (UID: \"3aedf792-6f12-4507-9585-1c0b1d74ccfa\") " pod="kuadrant-system/authorino-59c974bd48-c8w9r"
Sep 18 20:07:05.842928 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:05.842838    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-c8w9r"
Sep 18 20:07:06.132541 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:06.132510    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-c8w9r"]
Sep 18 20:07:06.134590 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:07:06.134567    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3aedf792_6f12_4507_9585_1c0b1d74ccfa.slice/crio-c05ab2e72b44b7eccb0a9f143a578932632cb7ba228fbfe93f02ceda226450bc WatchSource:0}: Error finding container c05ab2e72b44b7eccb0a9f143a578932632cb7ba228fbfe93f02ceda226450bc: Status 404 returned error can't find the container with id c05ab2e72b44b7eccb0a9f143a578932632cb7ba228fbfe93f02ceda226450bc
Sep 18 20:07:06.513276 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:06.513186    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-c8w9r" event={"ID":"3aedf792-6f12-4507-9585-1c0b1d74ccfa","Type":"ContainerStarted","Data":"c05ab2e72b44b7eccb0a9f143a578932632cb7ba228fbfe93f02ceda226450bc"}
Sep 18 20:07:07.520876 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:07.520785    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl" event={"ID":"043fcad7-4ad5-4ec3-bbae-680f7267db1e","Type":"ContainerStarted","Data":"3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f"}
Sep 18 20:07:07.521292 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:07.520920    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:07.542094 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:07.542038    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl" podStartSLOduration=1.398248493 podStartE2EDuration="3.542015319s" totalImagesPullingTime="2.143766826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:07:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:07:07.5397229 +0000 UTC m=+608.100306358" watchObservedRunningTime="2026-09-18 20:07:07.542015319 +0000 UTC m=+608.102598777"
Sep 18 20:07:08.527620 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:08.527586    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-c8w9r" event={"ID":"3aedf792-6f12-4507-9585-1c0b1d74ccfa","Type":"ContainerStarted","Data":"19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601"}
Sep 18 20:07:08.547106 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:08.547062    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-c8w9r" podStartSLOduration=1.622822966 podStartE2EDuration="3.547046774s" totalImagesPullingTime="1.924223808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:07:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:07:08.545020958 +0000 UTC m=+609.105604415" watchObservedRunningTime="2026-09-18 20:07:08.547046774 +0000 UTC m=+609.107630231"
Sep 18 20:07:17.454497 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:17.454461    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mpjrl"]
Sep 18 20:07:17.455101 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:17.454713    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl" podUID="043fcad7-4ad5-4ec3-bbae-680f7267db1e" containerName="limitador" containerID="cri-o://3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f" gracePeriod=30
Sep 18 20:07:17.455446 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:17.455354    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:18.000263 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.000236    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:18.160976 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.160945    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/043fcad7-4ad5-4ec3-bbae-680f7267db1e-config-file\") pod \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\" (UID: \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\") "
Sep 18 20:07:18.161137 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.161031    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd682\" (UniqueName: \"kubernetes.io/projected/043fcad7-4ad5-4ec3-bbae-680f7267db1e-kube-api-access-pd682\") pod \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\" (UID: \"043fcad7-4ad5-4ec3-bbae-680f7267db1e\") "
Sep 18 20:07:18.161306 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.161283    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/043fcad7-4ad5-4ec3-bbae-680f7267db1e-config-file" (OuterVolumeSpecName: "config-file") pod "043fcad7-4ad5-4ec3-bbae-680f7267db1e" (UID: "043fcad7-4ad5-4ec3-bbae-680f7267db1e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:07:18.163153 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.163125    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/043fcad7-4ad5-4ec3-bbae-680f7267db1e-kube-api-access-pd682" (OuterVolumeSpecName: "kube-api-access-pd682") pod "043fcad7-4ad5-4ec3-bbae-680f7267db1e" (UID: "043fcad7-4ad5-4ec3-bbae-680f7267db1e"). InnerVolumeSpecName "kube-api-access-pd682". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:07:18.262304 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.262274    2567 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/043fcad7-4ad5-4ec3-bbae-680f7267db1e-config-file\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:07:18.262304 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.262298    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd682\" (UniqueName: \"kubernetes.io/projected/043fcad7-4ad5-4ec3-bbae-680f7267db1e-kube-api-access-pd682\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:07:18.576742 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.576651    2567 generic.go:356] "Generic (PLEG): container finished" podID="043fcad7-4ad5-4ec3-bbae-680f7267db1e" containerID="3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f" exitCode=0
Sep 18 20:07:18.576742 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.576705    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl"
Sep 18 20:07:18.577181 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.576739    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl" event={"ID":"043fcad7-4ad5-4ec3-bbae-680f7267db1e","Type":"ContainerDied","Data":"3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f"}
Sep 18 20:07:18.577181 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.576791    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-mpjrl" event={"ID":"043fcad7-4ad5-4ec3-bbae-680f7267db1e","Type":"ContainerDied","Data":"f603cb379601eb0b9c171ae558b38cd6b9f68a930220e4bbd98e567a0346d7ca"}
Sep 18 20:07:18.577181 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.576814    2567 scope.go:117] "RemoveContainer" containerID="3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f"
Sep 18 20:07:18.591353 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.591328    2567 scope.go:117] "RemoveContainer" containerID="3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f"
Sep 18 20:07:18.591639 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:07:18.591615    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f\": container with ID starting with 3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f not found: ID does not exist" containerID="3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f"
Sep 18 20:07:18.591752 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.591646    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f"} err="failed to get container status \"3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f\": rpc error: code = NotFound desc = could not find container \"3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f\": container with ID starting with 3841bd1825c6120212cd0d50b5a52b8cd5796c357e566b7b208701432840f33f not found: ID does not exist"
Sep 18 20:07:18.603175 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.603150    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mpjrl"]
Sep 18 20:07:18.606744 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:18.606724    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-mpjrl"]
Sep 18 20:07:19.698671 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:19.698636    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="043fcad7-4ad5-4ec3-bbae-680f7267db1e" path="/var/lib/kubelet/pods/043fcad7-4ad5-4ec3-bbae-680f7267db1e/volumes"
Sep 18 20:07:35.942354 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:35.942317    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-c8w9r"]
Sep 18 20:07:35.942918 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:35.942518    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-c8w9r" podUID="3aedf792-6f12-4507-9585-1c0b1d74ccfa" containerName="authorino" containerID="cri-o://19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601" gracePeriod=30
Sep 18 20:07:36.187170 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.187141    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-c8w9r"
Sep 18 20:07:36.215208 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.215129    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tkpx\" (UniqueName: \"kubernetes.io/projected/3aedf792-6f12-4507-9585-1c0b1d74ccfa-kube-api-access-5tkpx\") pod \"3aedf792-6f12-4507-9585-1c0b1d74ccfa\" (UID: \"3aedf792-6f12-4507-9585-1c0b1d74ccfa\") "
Sep 18 20:07:36.217264 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.217238    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aedf792-6f12-4507-9585-1c0b1d74ccfa-kube-api-access-5tkpx" (OuterVolumeSpecName: "kube-api-access-5tkpx") pod "3aedf792-6f12-4507-9585-1c0b1d74ccfa" (UID: "3aedf792-6f12-4507-9585-1c0b1d74ccfa"). InnerVolumeSpecName "kube-api-access-5tkpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:07:36.316449 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.316417    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tkpx\" (UniqueName: \"kubernetes.io/projected/3aedf792-6f12-4507-9585-1c0b1d74ccfa-kube-api-access-5tkpx\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:07:36.662915 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.662878    2567 generic.go:356] "Generic (PLEG): container finished" podID="3aedf792-6f12-4507-9585-1c0b1d74ccfa" containerID="19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601" exitCode=0
Sep 18 20:07:36.663118 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.662922    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-c8w9r"
Sep 18 20:07:36.663118 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.662965    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-c8w9r" event={"ID":"3aedf792-6f12-4507-9585-1c0b1d74ccfa","Type":"ContainerDied","Data":"19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601"}
Sep 18 20:07:36.663118 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.663010    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-c8w9r" event={"ID":"3aedf792-6f12-4507-9585-1c0b1d74ccfa","Type":"ContainerDied","Data":"c05ab2e72b44b7eccb0a9f143a578932632cb7ba228fbfe93f02ceda226450bc"}
Sep 18 20:07:36.663118 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.663032    2567 scope.go:117] "RemoveContainer" containerID="19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601"
Sep 18 20:07:36.671578 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.671560    2567 scope.go:117] "RemoveContainer" containerID="19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601"
Sep 18 20:07:36.671934 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:07:36.671915    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601\": container with ID starting with 19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601 not found: ID does not exist" containerID="19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601"
Sep 18 20:07:36.672003 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.671946    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601"} err="failed to get container status \"19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601\": rpc error: code = NotFound desc = could not find container \"19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601\": container with ID starting with 19e003250004b88a2d853e18359f1cb69eda068710feafc365f09bd7af440601 not found: ID does not exist"
Sep 18 20:07:36.683480 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.683448    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-c8w9r"]
Sep 18 20:07:36.693215 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:36.689109    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-c8w9r"]
Sep 18 20:07:37.698437 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:37.698401    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3aedf792-6f12-4507-9585-1c0b1d74ccfa" path="/var/lib/kubelet/pods/3aedf792-6f12-4507-9585-1c0b1d74ccfa/volumes"
Sep 18 20:07:40.704324 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.704287    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-z2vjc"]
Sep 18 20:07:40.704929 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.704876    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="043fcad7-4ad5-4ec3-bbae-680f7267db1e" containerName="limitador"
Sep 18 20:07:40.704929 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.704921    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="043fcad7-4ad5-4ec3-bbae-680f7267db1e" containerName="limitador"
Sep 18 20:07:40.705074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.704934    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aedf792-6f12-4507-9585-1c0b1d74ccfa" containerName="authorino"
Sep 18 20:07:40.705074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.704942    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aedf792-6f12-4507-9585-1c0b1d74ccfa" containerName="authorino"
Sep 18 20:07:40.705178 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.705071    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="043fcad7-4ad5-4ec3-bbae-680f7267db1e" containerName="limitador"
Sep 18 20:07:40.705226 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.705185    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aedf792-6f12-4507-9585-1c0b1d74ccfa" containerName="authorino"
Sep 18 20:07:40.710944 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.710918    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.713216 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.713194    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 20:07:40.713346 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.713285    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j9h7q\""
Sep 18 20:07:40.713346 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.713296    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 20:07:40.713346 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.713307    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 20:07:40.717974 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.717950    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-z2vjc"]
Sep 18 20:07:40.769954 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.769918    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-metrics-certs\") pod \"maas-controller-b7fb6b9f7-z2vjc\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") " pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.770162 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.769966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhzmk\" (UniqueName: \"kubernetes.io/projected/057dc33d-1ceb-4535-8957-8b22867a8682-kube-api-access-mhzmk\") pod \"maas-controller-b7fb6b9f7-z2vjc\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") " pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.770162 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.770035    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-webhook-cert\") pod \"maas-controller-b7fb6b9f7-z2vjc\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") " pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.870882 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.870844    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-metrics-certs\") pod \"maas-controller-b7fb6b9f7-z2vjc\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") " pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.871072 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.870898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhzmk\" (UniqueName: \"kubernetes.io/projected/057dc33d-1ceb-4535-8957-8b22867a8682-kube-api-access-mhzmk\") pod \"maas-controller-b7fb6b9f7-z2vjc\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") " pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.871072 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.870962    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-webhook-cert\") pod \"maas-controller-b7fb6b9f7-z2vjc\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") " pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.872742 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.872709    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-988c79b4c-kgxbn"]
Sep 18 20:07:40.873373 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.873351    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-metrics-certs\") pod \"maas-controller-b7fb6b9f7-z2vjc\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") " pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.873620 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.873600    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-webhook-cert\") pod \"maas-controller-b7fb6b9f7-z2vjc\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") " pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.878702 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.878670    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:40.879929 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.879900    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhzmk\" (UniqueName: \"kubernetes.io/projected/057dc33d-1ceb-4535-8957-8b22867a8682-kube-api-access-mhzmk\") pod \"maas-controller-b7fb6b9f7-z2vjc\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") " pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:40.890528 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.890503    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-988c79b4c-kgxbn"]
Sep 18 20:07:40.972345 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.972255    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gndwk\" (UniqueName: \"kubernetes.io/projected/b638bdf6-e8fc-4111-980a-0acdeb104885-kube-api-access-gndwk\") pod \"maas-controller-988c79b4c-kgxbn\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") " pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:40.972345 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.972309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-webhook-cert\") pod \"maas-controller-988c79b4c-kgxbn\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") " pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:40.972532 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:40.972409    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-metrics-certs\") pod \"maas-controller-988c79b4c-kgxbn\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") " pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:41.024464 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.024432    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:41.073845 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.073813    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gndwk\" (UniqueName: \"kubernetes.io/projected/b638bdf6-e8fc-4111-980a-0acdeb104885-kube-api-access-gndwk\") pod \"maas-controller-988c79b4c-kgxbn\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") " pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:41.074032 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.073872    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-webhook-cert\") pod \"maas-controller-988c79b4c-kgxbn\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") " pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:41.074032 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.073949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-metrics-certs\") pod \"maas-controller-988c79b4c-kgxbn\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") " pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:41.076564 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.076534    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-metrics-certs\") pod \"maas-controller-988c79b4c-kgxbn\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") " pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:41.076681 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.076561    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-webhook-cert\") pod \"maas-controller-988c79b4c-kgxbn\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") " pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:41.083181 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.083143    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gndwk\" (UniqueName: \"kubernetes.io/projected/b638bdf6-e8fc-4111-980a-0acdeb104885-kube-api-access-gndwk\") pod \"maas-controller-988c79b4c-kgxbn\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") " pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:41.201363 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.201321    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:41.321627 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.321598    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-z2vjc"]
Sep 18 20:07:41.323567 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:07:41.323541    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057dc33d_1ceb_4535_8957_8b22867a8682.slice/crio-e18f553ddf1e85ff73993e47b89911ec620fcfc6dd1d7aa4fe48899f8b49c20c WatchSource:0}: Error finding container e18f553ddf1e85ff73993e47b89911ec620fcfc6dd1d7aa4fe48899f8b49c20c: Status 404 returned error can't find the container with id e18f553ddf1e85ff73993e47b89911ec620fcfc6dd1d7aa4fe48899f8b49c20c
Sep 18 20:07:41.502950 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.502926    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-988c79b4c-kgxbn"]
Sep 18 20:07:41.504748 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:07:41.504723    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb638bdf6_e8fc_4111_980a_0acdeb104885.slice/crio-ede85fec258c35369dd0ca7d15185239b73da90f7c8b1a60c8d4efca928e0f96 WatchSource:0}: Error finding container ede85fec258c35369dd0ca7d15185239b73da90f7c8b1a60c8d4efca928e0f96: Status 404 returned error can't find the container with id ede85fec258c35369dd0ca7d15185239b73da90f7c8b1a60c8d4efca928e0f96
Sep 18 20:07:41.691127 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.691087    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc" event={"ID":"057dc33d-1ceb-4535-8957-8b22867a8682","Type":"ContainerStarted","Data":"e18f553ddf1e85ff73993e47b89911ec620fcfc6dd1d7aa4fe48899f8b49c20c"}
Sep 18 20:07:41.692218 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:41.692193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-988c79b4c-kgxbn" event={"ID":"b638bdf6-e8fc-4111-980a-0acdeb104885","Type":"ContainerStarted","Data":"ede85fec258c35369dd0ca7d15185239b73da90f7c8b1a60c8d4efca928e0f96"}
Sep 18 20:07:44.710799 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:44.710752    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-988c79b4c-kgxbn" event={"ID":"b638bdf6-e8fc-4111-980a-0acdeb104885","Type":"ContainerStarted","Data":"59fcd9199097a5f57e3007e7fc6e9a8a317f0335e776f7156aa8f4cc43c31ab6"}
Sep 18 20:07:44.711548 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:44.710876    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:44.712303 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:44.712283    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc" event={"ID":"057dc33d-1ceb-4535-8957-8b22867a8682","Type":"ContainerStarted","Data":"b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a"}
Sep 18 20:07:44.712418 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:44.712403    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:44.730849 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:44.730799    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-988c79b4c-kgxbn" podStartSLOduration=1.980646023 podStartE2EDuration="4.730784136s" totalImagesPullingTime="2.750138113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:07:44.728859238 +0000 UTC m=+645.289442696" watchObservedRunningTime="2026-09-18 20:07:44.730784136 +0000 UTC m=+645.291367592"
Sep 18 20:07:44.799118 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:44.799066    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc" podStartSLOduration=1.855779426 podStartE2EDuration="4.799052169s" totalImagesPullingTime="2.943272743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:07:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:07:44.790353765 +0000 UTC m=+645.350937223" watchObservedRunningTime="2026-09-18 20:07:44.799052169 +0000 UTC m=+645.359635627"
Sep 18 20:07:55.719534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:55.719502    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:55.720017 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:55.719580    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:07:55.797806 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:55.797775    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-z2vjc"]
Sep 18 20:07:55.798030 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:55.798003    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc" podUID="057dc33d-1ceb-4535-8957-8b22867a8682" containerName="manager" containerID="cri-o://b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a" gracePeriod=30
Sep 18 20:07:56.037193 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.037169    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:56.116182 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.116151    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-webhook-cert\") pod \"057dc33d-1ceb-4535-8957-8b22867a8682\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") "
Sep 18 20:07:56.116346 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.116196    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-metrics-certs\") pod \"057dc33d-1ceb-4535-8957-8b22867a8682\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") "
Sep 18 20:07:56.116346 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.116243    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhzmk\" (UniqueName: \"kubernetes.io/projected/057dc33d-1ceb-4535-8957-8b22867a8682-kube-api-access-mhzmk\") pod \"057dc33d-1ceb-4535-8957-8b22867a8682\" (UID: \"057dc33d-1ceb-4535-8957-8b22867a8682\") "
Sep 18 20:07:56.118458 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.118422    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "057dc33d-1ceb-4535-8957-8b22867a8682" (UID: "057dc33d-1ceb-4535-8957-8b22867a8682"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:07:56.118602 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.118459    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "057dc33d-1ceb-4535-8957-8b22867a8682" (UID: "057dc33d-1ceb-4535-8957-8b22867a8682"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:07:56.118602 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.118508    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/057dc33d-1ceb-4535-8957-8b22867a8682-kube-api-access-mhzmk" (OuterVolumeSpecName: "kube-api-access-mhzmk") pod "057dc33d-1ceb-4535-8957-8b22867a8682" (UID: "057dc33d-1ceb-4535-8957-8b22867a8682"). InnerVolumeSpecName "kube-api-access-mhzmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:07:56.217277 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.217242    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhzmk\" (UniqueName: \"kubernetes.io/projected/057dc33d-1ceb-4535-8957-8b22867a8682-kube-api-access-mhzmk\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:07:56.217277 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.217272    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-webhook-cert\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:07:56.217277 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.217283    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/057dc33d-1ceb-4535-8957-8b22867a8682-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:07:56.771762 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.771727    2567 generic.go:356] "Generic (PLEG): container finished" podID="057dc33d-1ceb-4535-8957-8b22867a8682" containerID="b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a" exitCode=0
Sep 18 20:07:56.772156 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.771776    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc"
Sep 18 20:07:56.772156 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.771810    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc" event={"ID":"057dc33d-1ceb-4535-8957-8b22867a8682","Type":"ContainerDied","Data":"b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a"}
Sep 18 20:07:56.772156 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.771865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b7fb6b9f7-z2vjc" event={"ID":"057dc33d-1ceb-4535-8957-8b22867a8682","Type":"ContainerDied","Data":"e18f553ddf1e85ff73993e47b89911ec620fcfc6dd1d7aa4fe48899f8b49c20c"}
Sep 18 20:07:56.772156 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.771896    2567 scope.go:117] "RemoveContainer" containerID="b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a"
Sep 18 20:07:56.780809 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.780788    2567 scope.go:117] "RemoveContainer" containerID="b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a"
Sep 18 20:07:56.781090 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:07:56.781067    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a\": container with ID starting with b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a not found: ID does not exist" containerID="b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a"
Sep 18 20:07:56.781147 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.781100    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a"} err="failed to get container status \"b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a\": rpc error: code = NotFound desc = could not find container \"b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a\": container with ID starting with b2722e3befd110e88f76adef12acccac8c51b27b886d0ac565da74485ae0443a not found: ID does not exist"
Sep 18 20:07:56.793789 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.793767    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-z2vjc"]
Sep 18 20:07:56.800955 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:56.800934    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b7fb6b9f7-z2vjc"]
Sep 18 20:07:57.698950 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:07:57.698918    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="057dc33d-1ceb-4535-8957-8b22867a8682" path="/var/lib/kubelet/pods/057dc33d-1ceb-4535-8957-8b22867a8682/volumes"
Sep 18 20:08:16.586156 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.586110    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"]
Sep 18 20:08:16.586757 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.586576    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="057dc33d-1ceb-4535-8957-8b22867a8682" containerName="manager"
Sep 18 20:08:16.586757 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.586595    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="057dc33d-1ceb-4535-8957-8b22867a8682" containerName="manager"
Sep 18 20:08:16.586757 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.586700    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="057dc33d-1ceb-4535-8957-8b22867a8682" containerName="manager"
Sep 18 20:08:16.589740 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.589721    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.592524 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.592505    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 20:08:16.593090 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.593071    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 20:08:16.593204 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.593103    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 20:08:16.593204 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.593078    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:08:16.604994 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.604966    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"]
Sep 18 20:08:16.615477 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.615450    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rh2w\" (UniqueName: \"kubernetes.io/projected/f911de8f-d3db-4261-a12f-3a984f1c2921-kube-api-access-4rh2w\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.615596 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.615500    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911de8f-d3db-4261-a12f-3a984f1c2921-openshift-service-ca\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.615596 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.615565    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-maas-api-tls\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.615596 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.615591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-metrics-certs\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.716862 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.716815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-maas-api-tls\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.716862 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.716865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-metrics-certs\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.717099 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.716920    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rh2w\" (UniqueName: \"kubernetes.io/projected/f911de8f-d3db-4261-a12f-3a984f1c2921-kube-api-access-4rh2w\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.717099 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.716962    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911de8f-d3db-4261-a12f-3a984f1c2921-openshift-service-ca\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.717099 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:08:16.717057    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 18 20:08:16.717268 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:08:16.717137    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-metrics-certs podName:f911de8f-d3db-4261-a12f-3a984f1c2921 nodeName:}" failed. No retries permitted until 2026-09-18 20:08:17.217115871 +0000 UTC m=+677.777699320 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-metrics-certs") pod "maas-api-5598d8c9fb-lvzqf" (UID: "f911de8f-d3db-4261-a12f-3a984f1c2921") : secret "maas-api-metrics-tls" not found
Sep 18 20:08:16.717597 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.717580    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911de8f-d3db-4261-a12f-3a984f1c2921-openshift-service-ca\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.719597 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.719553    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-maas-api-tls\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:16.733335 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:16.733308    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rh2w\" (UniqueName: \"kubernetes.io/projected/f911de8f-d3db-4261-a12f-3a984f1c2921-kube-api-access-4rh2w\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:17.221667 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.221624    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-metrics-certs\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:17.224038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.224014    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-metrics-certs\") pod \"maas-api-5598d8c9fb-lvzqf\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:17.383774 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.383740    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"]
Sep 18 20:08:17.387400 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.387377    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:08:17.389647 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.389621    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 20:08:17.405231 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.405202    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"]
Sep 18 20:08:17.407341 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.407313    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"]
Sep 18 20:08:17.415242 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.415222    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:17.418245 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.418223    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-q8sd7\""
Sep 18 20:08:17.422632 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.422608    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"]
Sep 18 20:08:17.423479 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.423270    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0d168c9-2d92-420c-8187-833852d4c767-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ktlxz\" (UID: \"c0d168c9-2d92-420c-8187-833852d4c767\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:08:17.423479 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.423310    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhkr\" (UniqueName: \"kubernetes.io/projected/c0d168c9-2d92-420c-8187-833852d4c767-kube-api-access-9xhkr\") pod \"payload-pre-processing-59fbc47bf8-ktlxz\" (UID: \"c0d168c9-2d92-420c-8187-833852d4c767\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:08:17.500469 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.500380    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:17.524154 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.524123    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/056b9973-eb31-492f-b1d2-b2e3877abf2d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-xr9rk\" (UID: \"056b9973-eb31-492f-b1d2-b2e3877abf2d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:17.524286 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.524171    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqqtq\" (UniqueName: \"kubernetes.io/projected/056b9973-eb31-492f-b1d2-b2e3877abf2d-kube-api-access-rqqtq\") pod \"payload-processing-5b54b7fbb7-xr9rk\" (UID: \"056b9973-eb31-492f-b1d2-b2e3877abf2d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:17.524286 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.524206    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0d168c9-2d92-420c-8187-833852d4c767-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ktlxz\" (UID: \"c0d168c9-2d92-420c-8187-833852d4c767\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:08:17.524286 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.524229    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhkr\" (UniqueName: \"kubernetes.io/projected/c0d168c9-2d92-420c-8187-833852d4c767-kube-api-access-9xhkr\") pod \"payload-pre-processing-59fbc47bf8-ktlxz\" (UID: \"c0d168c9-2d92-420c-8187-833852d4c767\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:08:17.525338 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.525279    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0d168c9-2d92-420c-8187-833852d4c767-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-ktlxz\" (UID: \"c0d168c9-2d92-420c-8187-833852d4c767\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:08:17.534514 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.534487    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhkr\" (UniqueName: \"kubernetes.io/projected/c0d168c9-2d92-420c-8187-833852d4c767-kube-api-access-9xhkr\") pod \"payload-pre-processing-59fbc47bf8-ktlxz\" (UID: \"c0d168c9-2d92-420c-8187-833852d4c767\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:08:17.625440 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.625396    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/056b9973-eb31-492f-b1d2-b2e3877abf2d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-xr9rk\" (UID: \"056b9973-eb31-492f-b1d2-b2e3877abf2d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:17.625819 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.625460    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqqtq\" (UniqueName: \"kubernetes.io/projected/056b9973-eb31-492f-b1d2-b2e3877abf2d-kube-api-access-rqqtq\") pod \"payload-processing-5b54b7fbb7-xr9rk\" (UID: \"056b9973-eb31-492f-b1d2-b2e3877abf2d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:17.626178 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.626141    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/056b9973-eb31-492f-b1d2-b2e3877abf2d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-xr9rk\" (UID: \"056b9973-eb31-492f-b1d2-b2e3877abf2d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:17.635106 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.635073    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqqtq\" (UniqueName: \"kubernetes.io/projected/056b9973-eb31-492f-b1d2-b2e3877abf2d-kube-api-access-rqqtq\") pod \"payload-processing-5b54b7fbb7-xr9rk\" (UID: \"056b9973-eb31-492f-b1d2-b2e3877abf2d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:17.698597 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.698564    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:08:17.728176 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.728141    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:17.792514 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.792444    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"]
Sep 18 20:08:17.796212 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:08:17.796072    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf911de8f_d3db_4261_a12f_3a984f1c2921.slice/crio-7c271cc8bdc6d0a49f00a73d8d830bf7615149118fc41e198531bc03793e25c5 WatchSource:0}: Error finding container 7c271cc8bdc6d0a49f00a73d8d830bf7615149118fc41e198531bc03793e25c5: Status 404 returned error can't find the container with id 7c271cc8bdc6d0a49f00a73d8d830bf7615149118fc41e198531bc03793e25c5
Sep 18 20:08:17.872920 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.872884    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf" event={"ID":"f911de8f-d3db-4261-a12f-3a984f1c2921","Type":"ContainerStarted","Data":"7c271cc8bdc6d0a49f00a73d8d830bf7615149118fc41e198531bc03793e25c5"}
Sep 18 20:08:17.980662 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:17.980456    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"]
Sep 18 20:08:17.983511 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:08:17.983480    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d168c9_2d92_420c_8187_833852d4c767.slice/crio-65f1961ba7aae3984ac6a4f4da0347c1e8ace825b74e6bff3c4d6c0e6f26fdd9 WatchSource:0}: Error finding container 65f1961ba7aae3984ac6a4f4da0347c1e8ace825b74e6bff3c4d6c0e6f26fdd9: Status 404 returned error can't find the container with id 65f1961ba7aae3984ac6a4f4da0347c1e8ace825b74e6bff3c4d6c0e6f26fdd9
Sep 18 20:08:18.028248 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:18.028140    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"]
Sep 18 20:08:18.030959 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:08:18.030932    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056b9973_eb31_492f_b1d2_b2e3877abf2d.slice/crio-38e675f07b2f958c4095006b8492b835afcc285472a6ce184db1b235ab2f5932 WatchSource:0}: Error finding container 38e675f07b2f958c4095006b8492b835afcc285472a6ce184db1b235ab2f5932: Status 404 returned error can't find the container with id 38e675f07b2f958c4095006b8492b835afcc285472a6ce184db1b235ab2f5932
Sep 18 20:08:18.885941 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:18.885864    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz" event={"ID":"c0d168c9-2d92-420c-8187-833852d4c767","Type":"ContainerStarted","Data":"65f1961ba7aae3984ac6a4f4da0347c1e8ace825b74e6bff3c4d6c0e6f26fdd9"}
Sep 18 20:08:18.890255 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:18.890217    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk" event={"ID":"056b9973-eb31-492f-b1d2-b2e3877abf2d","Type":"ContainerStarted","Data":"38e675f07b2f958c4095006b8492b835afcc285472a6ce184db1b235ab2f5932"}
Sep 18 20:08:20.903895 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:20.903853    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk" event={"ID":"056b9973-eb31-492f-b1d2-b2e3877abf2d","Type":"ContainerStarted","Data":"f9b3c2fb9c69b0fb005de2f3456fa76780f3c93189ebb95a9dc6f14b532d9c22"}
Sep 18 20:08:20.904354 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:20.903924    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:20.905606 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:20.905572    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf" event={"ID":"f911de8f-d3db-4261-a12f-3a984f1c2921","Type":"ContainerStarted","Data":"28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8"}
Sep 18 20:08:20.905761 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:20.905642    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:20.907143 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:20.907121    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz" event={"ID":"c0d168c9-2d92-420c-8187-833852d4c767","Type":"ContainerStarted","Data":"32f576061225ac0c2890f3be9b5c2da4fe0b6ec79d5a1207e6d05198188a3eee"}
Sep 18 20:08:20.907271 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:20.907227    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:08:20.931702 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:20.929768    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk" podStartSLOduration=1.2666451539999999 podStartE2EDuration="3.929746895s" totalImagesPullingTime="2.663101741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:08:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:08:20.928722172 +0000 UTC m=+681.489305634" watchObservedRunningTime="2026-09-18 20:08:20.929746895 +0000 UTC m=+681.490330353"
Sep 18 20:08:20.952306 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:20.952214    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf" podStartSLOduration=2.083467324 podStartE2EDuration="4.952194428s" totalImagesPullingTime="2.868727104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:08:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:08:20.951580898 +0000 UTC m=+681.512164356" watchObservedRunningTime="2026-09-18 20:08:20.952194428 +0000 UTC m=+681.512777886"
Sep 18 20:08:20.973287 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:20.973132    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz" podStartSLOduration=1.2605539430000001 podStartE2EDuration="3.973113016s" totalImagesPullingTime="2.712559073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:08:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:08:20.971757825 +0000 UTC m=+681.532341282" watchObservedRunningTime="2026-09-18 20:08:20.973113016 +0000 UTC m=+681.533696474"
Sep 18 20:08:26.918963 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:26.918935    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:08:31.915140 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:31.915108    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-xr9rk"
Sep 18 20:08:31.915555 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:08:31.915189    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-ktlxz"
Sep 18 20:09:04.938998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:04.938923    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"]
Sep 18 20:09:04.945520 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:04.945499    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:04.951256 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:04.951231    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 18 20:09:04.951419 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:04.951232    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 20:09:04.951419 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:04.951232    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-c5rk2\""
Sep 18 20:09:04.953958 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:04.953937    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 20:09:04.959131 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:04.959108    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"]
Sep 18 20:09:05.081272 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.081231    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f531fdf6-7de5-4b73-9100-adfefc762b02-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.081450 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.081340    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.081450 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.081368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.081538 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.081452    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gfwz\" (UniqueName: \"kubernetes.io/projected/f531fdf6-7de5-4b73-9100-adfefc762b02-kube-api-access-4gfwz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.081538 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.081490    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.081538 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.081519    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.182619 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.182583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.182619 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.182627    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.182910 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.182668    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gfwz\" (UniqueName: \"kubernetes.io/projected/f531fdf6-7de5-4b73-9100-adfefc762b02-kube-api-access-4gfwz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.182910 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.182718    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.182910 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.182748    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.182910 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.182873    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f531fdf6-7de5-4b73-9100-adfefc762b02-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.183179 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.183159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.183240 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.183186    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.183240 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.183215    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.184991 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.184967    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f531fdf6-7de5-4b73-9100-adfefc762b02-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.185257 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.185238    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f531fdf6-7de5-4b73-9100-adfefc762b02-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.193084 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.193023    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gfwz\" (UniqueName: \"kubernetes.io/projected/f531fdf6-7de5-4b73-9100-adfefc762b02-kube-api-access-4gfwz\") pod \"e2e-distinct-simulated-kserve-59975d69f9-shtmh\" (UID: \"f531fdf6-7de5-4b73-9100-adfefc762b02\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.255611 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.255571    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:05.545000 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.544974    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"]
Sep 18 20:09:05.546395 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:09:05.546367    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf531fdf6_7de5_4b73_9100_adfefc762b02.slice/crio-57d867806b8be26017c0a8a9740f022e382ada844be1998b084501f946eeae93 WatchSource:0}: Error finding container 57d867806b8be26017c0a8a9740f022e382ada844be1998b084501f946eeae93: Status 404 returned error can't find the container with id 57d867806b8be26017c0a8a9740f022e382ada844be1998b084501f946eeae93
Sep 18 20:09:05.547969 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:05.547951    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:09:06.129366 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:06.129325    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh" event={"ID":"f531fdf6-7de5-4b73-9100-adfefc762b02","Type":"ContainerStarted","Data":"57d867806b8be26017c0a8a9740f022e382ada844be1998b084501f946eeae93"}
Sep 18 20:09:07.135221 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:07.135184    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh" event={"ID":"f531fdf6-7de5-4b73-9100-adfefc762b02","Type":"ContainerStarted","Data":"6d1f638709890be73129bf434ddef2cd8a6ba8cad28cffe67d8cd9c14b6a126b"}
Sep 18 20:09:07.162046 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:07.161996    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh" podStartSLOduration=2.150380057 podStartE2EDuration="3.161981127s" totalImagesPullingTime="1.01160107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:07.160271329 +0000 UTC m=+727.720854786" watchObservedRunningTime="2026-09-18 20:09:07.161981127 +0000 UTC m=+727.722564584"
Sep 18 20:09:14.615368 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.615334    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"]
Sep 18 20:09:14.618813 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.618792    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.621908 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.621885    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 18 20:09:14.633723 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.633695    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"]
Sep 18 20:09:14.774625 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.774574    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.774625 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.774626    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.774874 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.774833    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2nwn\" (UniqueName: \"kubernetes.io/projected/81fbef5d-e2d5-4008-962f-a67dd4cb2985-kube-api-access-h2nwn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.774913 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.774893    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.774945 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.774924    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.775021 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.775005    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81fbef5d-e2d5-4008-962f-a67dd4cb2985-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.875962 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.875865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81fbef5d-e2d5-4008-962f-a67dd4cb2985-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.875962 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.875944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.875962 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.875960    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.876192 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.876030    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2nwn\" (UniqueName: \"kubernetes.io/projected/81fbef5d-e2d5-4008-962f-a67dd4cb2985-kube-api-access-h2nwn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.876192 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.876060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.876192 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.876076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.876474 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.876448    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.876575 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.876479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.876575 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.876529    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.878337 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.878308    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81fbef5d-e2d5-4008-962f-a67dd4cb2985-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.878522 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.878505    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81fbef5d-e2d5-4008-962f-a67dd4cb2985-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.886509 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.886482    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2nwn\" (UniqueName: \"kubernetes.io/projected/81fbef5d-e2d5-4008-962f-a67dd4cb2985-kube-api-access-h2nwn\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf\" (UID: \"81fbef5d-e2d5-4008-962f-a67dd4cb2985\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:14.929136 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:14.929102    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:15.207487 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:15.207462    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"]
Sep 18 20:09:15.209991 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:09:15.209949    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81fbef5d_e2d5_4008_962f_a67dd4cb2985.slice/crio-7021b3389acd287c82953cf2b06c5f5bb343b711f0c521caca98d95ade2aa291 WatchSource:0}: Error finding container 7021b3389acd287c82953cf2b06c5f5bb343b711f0c521caca98d95ade2aa291: Status 404 returned error can't find the container with id 7021b3389acd287c82953cf2b06c5f5bb343b711f0c521caca98d95ade2aa291
Sep 18 20:09:15.256114 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:15.256078    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:15.256292 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:15.256140    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:15.268759 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:15.268719    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:16.181360 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:16.181323    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf" event={"ID":"81fbef5d-e2d5-4008-962f-a67dd4cb2985","Type":"ContainerStarted","Data":"2a4603b4a9f53d2505792e6c3c0c24029718fc8094d7e4c68410f7f0afd4d876"}
Sep 18 20:09:16.181861 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:16.181376    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf" event={"ID":"81fbef5d-e2d5-4008-962f-a67dd4cb2985","Type":"ContainerStarted","Data":"7021b3389acd287c82953cf2b06c5f5bb343b711f0c521caca98d95ade2aa291"}
Sep 18 20:09:16.192882 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:16.192851    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-shtmh"
Sep 18 20:09:16.202824 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:16.202781    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf" podStartSLOduration=2.019515434 podStartE2EDuration="2.202765971s" totalImagesPullingTime="183.250537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:16.200732278 +0000 UTC m=+736.761315735" watchObservedRunningTime="2026-09-18 20:09:16.202765971 +0000 UTC m=+736.763349484"
Sep 18 20:09:24.929972 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:24.929929    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:24.930390 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:24.929997    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:24.942780 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:24.942750    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:25.238177 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:25.238094    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf"
Sep 18 20:09:45.462951 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.462915    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"]
Sep 18 20:09:45.469178 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.469156    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.473322 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.473298    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 20:09:45.482907 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.482881    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"]
Sep 18 20:09:45.581874 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.581842    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.582038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.581883    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.582038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.581912    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rfss\" (UniqueName: \"kubernetes.io/projected/10b1bc9e-f555-4e8e-9118-052768bc3111-kube-api-access-9rfss\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.582038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.581960    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.582038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.582001    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b1bc9e-f555-4e8e-9118-052768bc3111-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.582176 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.582070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.683115 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.683072    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.683115 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.683117    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.683340 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.683148    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.683340 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.683190    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rfss\" (UniqueName: \"kubernetes.io/projected/10b1bc9e-f555-4e8e-9118-052768bc3111-kube-api-access-9rfss\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.683340 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.683221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.683340 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.683282    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b1bc9e-f555-4e8e-9118-052768bc3111-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.683627 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.683531    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.683718 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.683633    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.683780 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.683726    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.685510 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.685492    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10b1bc9e-f555-4e8e-9118-052768bc3111-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.685832 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.685816    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10b1bc9e-f555-4e8e-9118-052768bc3111-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.694089 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.694069    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rfss\" (UniqueName: \"kubernetes.io/projected/10b1bc9e-f555-4e8e-9118-052768bc3111-kube-api-access-9rfss\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-j9kj8\" (UID: \"10b1bc9e-f555-4e8e-9118-052768bc3111\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:45.781525 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:45.781436    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:46.064100 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:46.064073    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"]
Sep 18 20:09:46.066145 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:09:46.066119    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b1bc9e_f555_4e8e_9118_052768bc3111.slice/crio-97d4cf05dd46d7162915936abf231f035c78ef9236805d8b54b28891e331f030 WatchSource:0}: Error finding container 97d4cf05dd46d7162915936abf231f035c78ef9236805d8b54b28891e331f030: Status 404 returned error can't find the container with id 97d4cf05dd46d7162915936abf231f035c78ef9236805d8b54b28891e331f030
Sep 18 20:09:46.340360 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:46.340327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8" event={"ID":"10b1bc9e-f555-4e8e-9118-052768bc3111","Type":"ContainerStarted","Data":"21a56795bad01bb73a5d04ddcc9e9d1b5a39f40d781eac7750cee3794df71a07"}
Sep 18 20:09:46.340502 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:46.340377    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8" event={"ID":"10b1bc9e-f555-4e8e-9118-052768bc3111","Type":"ContainerStarted","Data":"97d4cf05dd46d7162915936abf231f035c78ef9236805d8b54b28891e331f030"}
Sep 18 20:09:46.358736 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:46.358669    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8" podStartSLOduration=1.15964523 podStartE2EDuration="1.358649329s" totalImagesPullingTime="199.004099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:46.357468396 +0000 UTC m=+766.918051900" watchObservedRunningTime="2026-09-18 20:09:46.358649329 +0000 UTC m=+766.919232786"
Sep 18 20:09:51.637640 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.637608    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"]
Sep 18 20:09:51.641202 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.641183    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.644903 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.644883    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 18 20:09:51.677337 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.677307    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"]
Sep 18 20:09:51.742157 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.742121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.742157 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.742163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.742364 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.742190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.742364 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.742236    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqqs8\" (UniqueName: \"kubernetes.io/projected/a3019236-d88e-48e8-9efb-27eb14631179-kube-api-access-wqqs8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.742364 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.742322    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.742471 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.742364    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3019236-d88e-48e8-9efb-27eb14631179-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.843653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.843612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.843653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.843654    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.843927 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.843718    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.843927 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.843737    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqqs8\" (UniqueName: \"kubernetes.io/projected/a3019236-d88e-48e8-9efb-27eb14631179-kube-api-access-wqqs8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.843927 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.843756    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.843927 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.843865    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3019236-d88e-48e8-9efb-27eb14631179-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.844133 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.844105    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.844196 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.844145    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.844196 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.844182    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.846281 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.846259    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3019236-d88e-48e8-9efb-27eb14631179-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.846446 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.846427    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3019236-d88e-48e8-9efb-27eb14631179-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.854318 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.854293    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqqs8\" (UniqueName: \"kubernetes.io/projected/a3019236-d88e-48e8-9efb-27eb14631179-kube-api-access-wqqs8\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd\" (UID: \"a3019236-d88e-48e8-9efb-27eb14631179\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:51.950926 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:51.950840    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:09:52.233052 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:52.233023    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"]
Sep 18 20:09:52.235043 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:09:52.235016    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3019236_d88e_48e8_9efb_27eb14631179.slice/crio-8f73bc6c1469bfc876447f9926ab29b55fbf5ea09c69c344169287e922d54e8d WatchSource:0}: Error finding container 8f73bc6c1469bfc876447f9926ab29b55fbf5ea09c69c344169287e922d54e8d: Status 404 returned error can't find the container with id 8f73bc6c1469bfc876447f9926ab29b55fbf5ea09c69c344169287e922d54e8d
Sep 18 20:09:52.371860 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:52.371830    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd" event={"ID":"a3019236-d88e-48e8-9efb-27eb14631179","Type":"ContainerStarted","Data":"8f73bc6c1469bfc876447f9926ab29b55fbf5ea09c69c344169287e922d54e8d"}
Sep 18 20:09:53.378736 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:53.378667    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd" event={"ID":"a3019236-d88e-48e8-9efb-27eb14631179","Type":"ContainerStarted","Data":"d515fdbcb61e29bc008611bfb2aeb2c9bcbfa1d0b2261ebfa05c7fdf95f3e80c"}
Sep 18 20:09:53.407349 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:53.407295    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd" podStartSLOduration=2.2281903 podStartE2EDuration="2.407271621s" totalImagesPullingTime="179.081321ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:53.405187399 +0000 UTC m=+773.965770855" watchObservedRunningTime="2026-09-18 20:09:53.407271621 +0000 UTC m=+773.967855081"
Sep 18 20:09:55.782371 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:55.782333    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:55.782788 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:55.782415    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:55.795336 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:55.795307    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:09:56.406649 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:09:56.406619    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-j9kj8"
Sep 18 20:10:01.950965 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:01.950929    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:10:01.951374 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:01.950982    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:10:01.963632 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:01.963603    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:10:02.437882 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:02.437846    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd"
Sep 18 20:10:56.437654 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.437619    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b6fdfb5bb-n8vdb"]
Sep 18 20:10:56.441045 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.441027    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb"
Sep 18 20:10:56.446918 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.446889    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 18 20:10:56.447069 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.446889    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nhv9b\""
Sep 18 20:10:56.447069 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.446902    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 20:10:56.447280 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.447263    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 20:10:56.460310 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.460275    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b6fdfb5bb-n8vdb"]
Sep 18 20:10:56.622336 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.622295    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757-tls-cert\") pod \"authorino-5b6fdfb5bb-n8vdb\" (UID: \"1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757\") " pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb"
Sep 18 20:10:56.622513 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.622350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5hw7\" (UniqueName: \"kubernetes.io/projected/1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757-kube-api-access-t5hw7\") pod \"authorino-5b6fdfb5bb-n8vdb\" (UID: \"1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757\") " pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb"
Sep 18 20:10:56.723909 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.723820    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757-tls-cert\") pod \"authorino-5b6fdfb5bb-n8vdb\" (UID: \"1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757\") " pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb"
Sep 18 20:10:56.723909 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.723880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5hw7\" (UniqueName: \"kubernetes.io/projected/1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757-kube-api-access-t5hw7\") pod \"authorino-5b6fdfb5bb-n8vdb\" (UID: \"1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757\") " pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb"
Sep 18 20:10:56.726459 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.726425    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757-tls-cert\") pod \"authorino-5b6fdfb5bb-n8vdb\" (UID: \"1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757\") " pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb"
Sep 18 20:10:56.743485 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.743458    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5hw7\" (UniqueName: \"kubernetes.io/projected/1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757-kube-api-access-t5hw7\") pod \"authorino-5b6fdfb5bb-n8vdb\" (UID: \"1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757\") " pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb"
Sep 18 20:10:56.750337 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:56.750305    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb"
Sep 18 20:10:57.042465 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:57.042379    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b6fdfb5bb-n8vdb"]
Sep 18 20:10:57.046384 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:10:57.046355    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f5c9f7d_fb9f_43a1_8743_e9d50ad3a757.slice/crio-54ee5bdf0647303717dac7f6c601dedc842eb94ddf726791b5be3bee613aca58 WatchSource:0}: Error finding container 54ee5bdf0647303717dac7f6c601dedc842eb94ddf726791b5be3bee613aca58: Status 404 returned error can't find the container with id 54ee5bdf0647303717dac7f6c601dedc842eb94ddf726791b5be3bee613aca58
Sep 18 20:10:57.714634 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:57.714596    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb" event={"ID":"1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757","Type":"ContainerStarted","Data":"54ee5bdf0647303717dac7f6c601dedc842eb94ddf726791b5be3bee613aca58"}
Sep 18 20:10:58.721601 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:58.721565    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb" event={"ID":"1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757","Type":"ContainerStarted","Data":"2430944b68855183bbba5ba13961750a31d54e3971c4d70dfd6dfdca5ed3c5b1"}
Sep 18 20:10:58.765317 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:10:58.765246    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5b6fdfb5bb-n8vdb" podStartSLOduration=2.059041817 podStartE2EDuration="2.765222056s" totalImagesPullingTime="706.180239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:10:58.763119711 +0000 UTC m=+839.323703169" watchObservedRunningTime="2026-09-18 20:10:58.765222056 +0000 UTC m=+839.325805516"
Sep 18 20:11:26.563910 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:26.563866    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"]
Sep 18 20:11:26.567784 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:26.567759    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:26.577627 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:11:26.577592    2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm\" is forbidden: User \"system:node:ip-10-0-130-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-248.ec2.internal' and this object" podUID="e46fa62e-d8dd-4d48-8091-b5eda92236a3" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:26.577627 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:11:26.577601    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"payload-processing-plugins-e2e-worker-w0-5284dc\" is forbidden: User \"system:node:ip-10-0-130-248.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-248.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-5284dc\"" type="*v1.ConfigMap"
Sep 18 20:11:26.577788 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:11:26.577601    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-e2e-worker-w0-5284dc-dockercfg-g5v72\" is forbidden: User \"system:node:ip-10-0-130-248.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-248.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-5284dc-dockercfg-g5v72\"" type="*v1.Secret"
Sep 18 20:11:26.630258 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:26.630225    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"]
Sep 18 20:11:26.724856 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:26.724816    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e46fa62e-d8dd-4d48-8091-b5eda92236a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm\" (UID: \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:26.725001 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:26.724933    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjttx\" (UniqueName: \"kubernetes.io/projected/e46fa62e-d8dd-4d48-8091-b5eda92236a3-kube-api-access-hjttx\") pod \"payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm\" (UID: \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:26.825926 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:26.825836    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjttx\" (UniqueName: \"kubernetes.io/projected/e46fa62e-d8dd-4d48-8091-b5eda92236a3-kube-api-access-hjttx\") pod \"payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm\" (UID: \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:26.826117 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:26.826088    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e46fa62e-d8dd-4d48-8091-b5eda92236a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm\" (UID: \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:26.844792 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:26.844757    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjttx\" (UniqueName: \"kubernetes.io/projected/e46fa62e-d8dd-4d48-8091-b5eda92236a3-kube-api-access-hjttx\") pod \"payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm\" (UID: \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:27.135792 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.135760    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"]
Sep 18 20:11:27.139336 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.139318    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.147860 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.147831    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-4b633d-serving-cert\""
Sep 18 20:11:27.148115 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.147915    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kcjh\""
Sep 18 20:11:27.215613 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.215577    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"]
Sep 18 20:11:27.229542 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.229502    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/620af298-7d5b-4723-9365-12e31947e124-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.229732 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.229641    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q6z5\" (UniqueName: \"kubernetes.io/projected/620af298-7d5b-4723-9365-12e31947e124-kube-api-access-7q6z5\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.229732 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.229674    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-metrics-certs\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.229874 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.229771    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-maas-api-tls\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.330537 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.330496    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/620af298-7d5b-4723-9365-12e31947e124-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.330752 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.330570    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q6z5\" (UniqueName: \"kubernetes.io/projected/620af298-7d5b-4723-9365-12e31947e124-kube-api-access-7q6z5\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.330752 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.330593    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-metrics-certs\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.330752 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.330626    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-maas-api-tls\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.331463 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.331433    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/620af298-7d5b-4723-9365-12e31947e124-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.333441 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.333415    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-maas-api-tls\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.333548 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.333480    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-metrics-certs\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.359279 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.359248    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q6z5\" (UniqueName: \"kubernetes.io/projected/620af298-7d5b-4723-9365-12e31947e124-kube-api-access-7q6z5\") pod \"maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.451480 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.451393    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:27.513594 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.513323    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-5284dc-dockercfg-g5v72\""
Sep 18 20:11:27.804876 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:11:27.804834    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod620af298_7d5b_4723_9365_12e31947e124.slice/crio-09d2add378311500aa29649cadf912645aef88a1006594ab08517e4b1ef6656d WatchSource:0}: Error finding container 09d2add378311500aa29649cadf912645aef88a1006594ab08517e4b1ef6656d: Status 404 returned error can't find the container with id 09d2add378311500aa29649cadf912645aef88a1006594ab08517e4b1ef6656d
Sep 18 20:11:27.811807 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.811777    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"]
Sep 18 20:11:27.826522 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:11:27.826490    2567 configmap.go:193] Couldn't get configMap openshift-ingress/payload-processing-plugins-e2e-worker-w0-5284dc: failed to sync configmap cache: timed out waiting for the condition
Sep 18 20:11:27.826712 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:11:27.826627    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e46fa62e-d8dd-4d48-8091-b5eda92236a3-plugins-config-volume podName:e46fa62e-d8dd-4d48-8091-b5eda92236a3 nodeName:}" failed. No retries permitted until 2026-09-18 20:11:28.326588355 +0000 UTC m=+868.887171805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/e46fa62e-d8dd-4d48-8091-b5eda92236a3-plugins-config-volume") pod "payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm" (UID: "e46fa62e-d8dd-4d48-8091-b5eda92236a3") : failed to sync configmap cache: timed out waiting for the condition
Sep 18 20:11:27.886793 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.886742    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj" event={"ID":"620af298-7d5b-4723-9365-12e31947e124","Type":"ContainerStarted","Data":"09d2add378311500aa29649cadf912645aef88a1006594ab08517e4b1ef6656d"}
Sep 18 20:11:27.991900 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:27.991869    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-5284dc\""
Sep 18 20:11:28.340158 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.340117    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e46fa62e-d8dd-4d48-8091-b5eda92236a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm\" (UID: \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:28.340856 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.340836    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e46fa62e-d8dd-4d48-8091-b5eda92236a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm\" (UID: \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:28.378133 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.378096    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:28.693638 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.693611    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"]
Sep 18 20:11:28.696155 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:11:28.696117    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46fa62e_d8dd_4d48_8091_b5eda92236a3.slice/crio-d93c25cdf71921b4d2c167d00f26c9afd60bea4449e77fe360708e3b9e503156 WatchSource:0}: Error finding container d93c25cdf71921b4d2c167d00f26c9afd60bea4449e77fe360708e3b9e503156: Status 404 returned error can't find the container with id d93c25cdf71921b4d2c167d00f26c9afd60bea4449e77fe360708e3b9e503156
Sep 18 20:11:28.893411 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.893346    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj" event={"ID":"620af298-7d5b-4723-9365-12e31947e124","Type":"ContainerStarted","Data":"e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c"}
Sep 18 20:11:28.893875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.893602    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:28.894946 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.894924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm" event={"ID":"e46fa62e-d8dd-4d48-8091-b5eda92236a3","Type":"ContainerStarted","Data":"dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef"}
Sep 18 20:11:28.895084 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.894960    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm" event={"ID":"e46fa62e-d8dd-4d48-8091-b5eda92236a3","Type":"ContainerStarted","Data":"d93c25cdf71921b4d2c167d00f26c9afd60bea4449e77fe360708e3b9e503156"}
Sep 18 20:11:28.895084 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.895044    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:28.953310 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:28.953209    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj" podStartSLOduration=1.5352195 podStartE2EDuration="1.953190067s" totalImagesPullingTime="417.970567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:28.947380795 +0000 UTC m=+869.507964265" watchObservedRunningTime="2026-09-18 20:11:28.953190067 +0000 UTC m=+869.513773525"
Sep 18 20:11:29.054791 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.054732    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm" podStartSLOduration=3.054713486 podStartE2EDuration="3.054713486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:29.053317654 +0000 UTC m=+869.613901112" watchObservedRunningTime="2026-09-18 20:11:29.054713486 +0000 UTC m=+869.615296942"
Sep 18 20:11:29.223278 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.223190    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"]
Sep 18 20:11:29.226987 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.226964    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:29.234972 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.234941    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-4b633d-dockercfg-qrb5x\""
Sep 18 20:11:29.235154 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.234971    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-4b633d\""
Sep 18 20:11:29.253940 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.253906    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"]
Sep 18 20:11:29.351870 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.351829    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e21e30ad-89e2-474d-8614-6b330628eecb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq\" (UID: \"e21e30ad-89e2-474d-8614-6b330628eecb\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:29.352069 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.351894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg6ss\" (UniqueName: \"kubernetes.io/projected/e21e30ad-89e2-474d-8614-6b330628eecb-kube-api-access-tg6ss\") pod \"payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq\" (UID: \"e21e30ad-89e2-474d-8614-6b330628eecb\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:29.452660 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.452622    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e21e30ad-89e2-474d-8614-6b330628eecb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq\" (UID: \"e21e30ad-89e2-474d-8614-6b330628eecb\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:29.452660 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.452670    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg6ss\" (UniqueName: \"kubernetes.io/projected/e21e30ad-89e2-474d-8614-6b330628eecb-kube-api-access-tg6ss\") pod \"payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq\" (UID: \"e21e30ad-89e2-474d-8614-6b330628eecb\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:29.453502 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.453479    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e21e30ad-89e2-474d-8614-6b330628eecb-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq\" (UID: \"e21e30ad-89e2-474d-8614-6b330628eecb\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:29.502677 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.502599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg6ss\" (UniqueName: \"kubernetes.io/projected/e21e30ad-89e2-474d-8614-6b330628eecb-kube-api-access-tg6ss\") pod \"payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq\" (UID: \"e21e30ad-89e2-474d-8614-6b330628eecb\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:29.540213 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.540177    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:29.849236 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.849200    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"]
Sep 18 20:11:29.850070 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:11:29.850030    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode21e30ad_89e2_474d_8614_6b330628eecb.slice/crio-1bcbbdebeb73442e8b65ddbdc2335d02fa55095d39a38940b67dd2f3fe5e7fee WatchSource:0}: Error finding container 1bcbbdebeb73442e8b65ddbdc2335d02fa55095d39a38940b67dd2f3fe5e7fee: Status 404 returned error can't find the container with id 1bcbbdebeb73442e8b65ddbdc2335d02fa55095d39a38940b67dd2f3fe5e7fee
Sep 18 20:11:29.902218 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:29.902181    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq" event={"ID":"e21e30ad-89e2-474d-8614-6b330628eecb","Type":"ContainerStarted","Data":"1bcbbdebeb73442e8b65ddbdc2335d02fa55095d39a38940b67dd2f3fe5e7fee"}
Sep 18 20:11:30.909217 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:30.909176    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq" event={"ID":"e21e30ad-89e2-474d-8614-6b330628eecb","Type":"ContainerStarted","Data":"5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde"}
Sep 18 20:11:30.909600 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:30.909252    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:30.950594 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:30.950522    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq" podStartSLOduration=1.95049785 podStartE2EDuration="1.95049785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:30.947383744 +0000 UTC m=+871.507967201" watchObservedRunningTime="2026-09-18 20:11:30.95049785 +0000 UTC m=+871.511081308"
Sep 18 20:11:31.854620 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:31.854585    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"]
Sep 18 20:11:31.858213 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:31.858184    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:31.861078 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:31.861052    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-3a8052-dockercfg-9gqwz\""
Sep 18 20:11:31.862449 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:31.862429    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-3a8052\""
Sep 18 20:11:31.873583 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:31.873556    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"]
Sep 18 20:11:31.980550 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:31.980502    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe658fa-e679-409e-b5a3-00fca0138cb6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd\" (UID: \"ebe658fa-e679-409e-b5a3-00fca0138cb6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:31.980550 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:31.980551    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jdsv\" (UniqueName: \"kubernetes.io/projected/ebe658fa-e679-409e-b5a3-00fca0138cb6-kube-api-access-6jdsv\") pod \"payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd\" (UID: \"ebe658fa-e679-409e-b5a3-00fca0138cb6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:32.081987 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.081941    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe658fa-e679-409e-b5a3-00fca0138cb6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd\" (UID: \"ebe658fa-e679-409e-b5a3-00fca0138cb6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:32.081987 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.081989    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jdsv\" (UniqueName: \"kubernetes.io/projected/ebe658fa-e679-409e-b5a3-00fca0138cb6-kube-api-access-6jdsv\") pod \"payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd\" (UID: \"ebe658fa-e679-409e-b5a3-00fca0138cb6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:32.082677 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.082653    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe658fa-e679-409e-b5a3-00fca0138cb6-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd\" (UID: \"ebe658fa-e679-409e-b5a3-00fca0138cb6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:32.100664 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.100631    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jdsv\" (UniqueName: \"kubernetes.io/projected/ebe658fa-e679-409e-b5a3-00fca0138cb6-kube-api-access-6jdsv\") pod \"payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd\" (UID: \"ebe658fa-e679-409e-b5a3-00fca0138cb6\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:32.169813 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.169670    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:32.461314 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.461276    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"]
Sep 18 20:11:32.463098 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:11:32.463061    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe658fa_e679_409e_b5a3_00fca0138cb6.slice/crio-b9419706612cc0884b1b48827e65b772d1c6df106c8d889f6dc5ca22e5176d18 WatchSource:0}: Error finding container b9419706612cc0884b1b48827e65b772d1c6df106c8d889f6dc5ca22e5176d18: Status 404 returned error can't find the container with id b9419706612cc0884b1b48827e65b772d1c6df106c8d889f6dc5ca22e5176d18
Sep 18 20:11:32.921425 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.921384    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd" event={"ID":"ebe658fa-e679-409e-b5a3-00fca0138cb6","Type":"ContainerStarted","Data":"f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7"}
Sep 18 20:11:32.921607 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.921445    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd" event={"ID":"ebe658fa-e679-409e-b5a3-00fca0138cb6","Type":"ContainerStarted","Data":"b9419706612cc0884b1b48827e65b772d1c6df106c8d889f6dc5ca22e5176d18"}
Sep 18 20:11:32.921607 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.921469    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:32.956021 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:32.955958    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd" podStartSLOduration=1.955935151 podStartE2EDuration="1.955935151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:32.955020168 +0000 UTC m=+873.515603626" watchObservedRunningTime="2026-09-18 20:11:32.955935151 +0000 UTC m=+873.516518611"
Sep 18 20:11:34.908749 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:34.908722    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:11:39.252059 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.252018    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"]
Sep 18 20:11:39.256123 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.256096    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.258963 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.258926    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nxrx\" (UniqueName: \"kubernetes.io/projected/28eb230e-55ef-44e2-9676-7eb850504065-kube-api-access-2nxrx\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.259108 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.258980    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28eb230e-55ef-44e2-9676-7eb850504065-openshift-service-ca\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.259108 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.259096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-maas-api-tls\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.259194 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.259121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-metrics-certs\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.262667 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.262646    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-d0d77c-serving-cert\""
Sep 18 20:11:39.288841 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.288752    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"]
Sep 18 20:11:39.359699 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.359642    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nxrx\" (UniqueName: \"kubernetes.io/projected/28eb230e-55ef-44e2-9676-7eb850504065-kube-api-access-2nxrx\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.359898 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.359712    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28eb230e-55ef-44e2-9676-7eb850504065-openshift-service-ca\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.359898 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.359785    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-maas-api-tls\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.359898 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.359808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-metrics-certs\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.360449 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.360424    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28eb230e-55ef-44e2-9676-7eb850504065-openshift-service-ca\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.362354 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.362328    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-metrics-certs\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.362492 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.362471    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-maas-api-tls\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.404078 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.404046    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nxrx\" (UniqueName: \"kubernetes.io/projected/28eb230e-55ef-44e2-9676-7eb850504065-kube-api-access-2nxrx\") pod \"maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.598908 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.598876    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:39.905071 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:39.904972    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:11:40.207422 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:40.207331    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"]
Sep 18 20:11:40.215357 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:11:40.215319    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28eb230e_55ef_44e2_9676_7eb850504065.slice/crio-a28ed816d2715ef198cf1e587506dabd4c26fa688c0367e03d834e75c3315da0 WatchSource:0}: Error finding container a28ed816d2715ef198cf1e587506dabd4c26fa688c0367e03d834e75c3315da0: Status 404 returned error can't find the container with id a28ed816d2715ef198cf1e587506dabd4c26fa688c0367e03d834e75c3315da0
Sep 18 20:11:40.971125 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:40.971025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv" event={"ID":"28eb230e-55ef-44e2-9676-7eb850504065","Type":"ContainerStarted","Data":"65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb"}
Sep 18 20:11:40.971125 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:40.971085    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv" event={"ID":"28eb230e-55ef-44e2-9676-7eb850504065","Type":"ContainerStarted","Data":"a28ed816d2715ef198cf1e587506dabd4c26fa688c0367e03d834e75c3315da0"}
Sep 18 20:11:40.971636 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:40.971173    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:41.917873 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:41.917841    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:11:41.985860 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:41.985805    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv" podStartSLOduration=2.5969191179999997 podStartE2EDuration="2.985786259s" totalImagesPullingTime="388.867141ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:41.216514385 +0000 UTC m=+881.777097841" watchObservedRunningTime="2026-09-18 20:11:41.985786259 +0000 UTC m=+882.546369716"
Sep 18 20:11:43.929303 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:43.929223    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:11:46.984397 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:46.984366    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:11:49.666000 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.665760    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"]
Sep 18 20:11:49.671578 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.670818    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:11:49.673669 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.673636    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-0a7e3fae\""
Sep 18 20:11:49.673853 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.673768    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-0a7e3fae-dockercfg-s5kl8\""
Sep 18 20:11:49.686373 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.686341    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"]
Sep 18 20:11:49.784916 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.784874    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnd4x\" (UniqueName: \"kubernetes.io/projected/b7b3722a-3622-4a36-91ca-7224ca51eae8-kube-api-access-jnd4x\") pod \"payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs\" (UID: \"b7b3722a-3622-4a36-91ca-7224ca51eae8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:11:49.785087 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.785050    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b3722a-3622-4a36-91ca-7224ca51eae8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs\" (UID: \"b7b3722a-3622-4a36-91ca-7224ca51eae8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:11:49.886200 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.886161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnd4x\" (UniqueName: \"kubernetes.io/projected/b7b3722a-3622-4a36-91ca-7224ca51eae8-kube-api-access-jnd4x\") pod \"payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs\" (UID: \"b7b3722a-3622-4a36-91ca-7224ca51eae8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:11:49.886200 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.886212    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b3722a-3622-4a36-91ca-7224ca51eae8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs\" (UID: \"b7b3722a-3622-4a36-91ca-7224ca51eae8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:11:49.887068 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.887023    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b3722a-3622-4a36-91ca-7224ca51eae8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs\" (UID: \"b7b3722a-3622-4a36-91ca-7224ca51eae8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:11:49.896714 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.896652    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnd4x\" (UniqueName: \"kubernetes.io/projected/b7b3722a-3622-4a36-91ca-7224ca51eae8-kube-api-access-jnd4x\") pod \"payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs\" (UID: \"b7b3722a-3622-4a36-91ca-7224ca51eae8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:11:49.984841 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:49.984741    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:11:50.265552 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:50.265523    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"]
Sep 18 20:11:50.267658 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:11:50.267629    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b3722a_3622_4a36_91ca_7224ca51eae8.slice/crio-370f1af4467ddfcf69ba08367e9fb26ebd78a4ba4e3657291fbbe5a0d94e6c01 WatchSource:0}: Error finding container 370f1af4467ddfcf69ba08367e9fb26ebd78a4ba4e3657291fbbe5a0d94e6c01: Status 404 returned error can't find the container with id 370f1af4467ddfcf69ba08367e9fb26ebd78a4ba4e3657291fbbe5a0d94e6c01
Sep 18 20:11:51.031013 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:51.030915    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs" event={"ID":"b7b3722a-3622-4a36-91ca-7224ca51eae8","Type":"ContainerStarted","Data":"75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4"}
Sep 18 20:11:51.031013 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:51.030990    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:11:51.031013 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:51.031016    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs" event={"ID":"b7b3722a-3622-4a36-91ca-7224ca51eae8","Type":"ContainerStarted","Data":"370f1af4467ddfcf69ba08367e9fb26ebd78a4ba4e3657291fbbe5a0d94e6c01"}
Sep 18 20:11:51.071653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:51.071595    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs" podStartSLOduration=2.071576545 podStartE2EDuration="2.071576545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:51.068348233 +0000 UTC m=+891.628931712" watchObservedRunningTime="2026-09-18 20:11:51.071576545 +0000 UTC m=+891.632160018"
Sep 18 20:11:55.915591 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:55.915548    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"]
Sep 18 20:11:55.920585 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:55.920541    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:55.923351 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:55.923325    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c34fad-serving-cert\""
Sep 18 20:11:55.933933 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:55.933891    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"]
Sep 18 20:11:56.053775 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.053734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq8d2\" (UniqueName: \"kubernetes.io/projected/83f8e21b-264a-4a6f-a823-b6878475b2cd-kube-api-access-jq8d2\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.053961 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.053884    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-metrics-certs\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.053961 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.053936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-maas-api-tls\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.054069 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.054050    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83f8e21b-264a-4a6f-a823-b6878475b2cd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.155115 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.155080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83f8e21b-264a-4a6f-a823-b6878475b2cd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.155115 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.155122    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq8d2\" (UniqueName: \"kubernetes.io/projected/83f8e21b-264a-4a6f-a823-b6878475b2cd-kube-api-access-jq8d2\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.155358 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.155178    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-metrics-certs\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.155358 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.155208    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-maas-api-tls\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.155804 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.155776    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83f8e21b-264a-4a6f-a823-b6878475b2cd-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.157873 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.157846    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-metrics-certs\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.157968 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.157883    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-maas-api-tls\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.165521 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.165494    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq8d2\" (UniqueName: \"kubernetes.io/projected/83f8e21b-264a-4a6f-a823-b6878475b2cd-kube-api-access-jq8d2\") pod \"maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.234645 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.234542    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:56.513801 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:56.513769    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"]
Sep 18 20:11:56.516083 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:11:56.516047    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f8e21b_264a_4a6f_a823_b6878475b2cd.slice/crio-16b39f81b7fe0a81354130cf09a357e0ab062f15d806600093373393d8468317 WatchSource:0}: Error finding container 16b39f81b7fe0a81354130cf09a357e0ab062f15d806600093373393d8468317: Status 404 returned error can't find the container with id 16b39f81b7fe0a81354130cf09a357e0ab062f15d806600093373393d8468317
Sep 18 20:11:57.028334 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.028247    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"]
Sep 18 20:11:57.032898 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.032870    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:11:57.035954 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.035929    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-c34fad-dockercfg-r7f42\""
Sep 18 20:11:57.036456 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.036434    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-c34fad\""
Sep 18 20:11:57.048600 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.048570    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"]
Sep 18 20:11:57.067575 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.067541    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x" event={"ID":"83f8e21b-264a-4a6f-a823-b6878475b2cd","Type":"ContainerStarted","Data":"d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674"}
Sep 18 20:11:57.067768 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.067602    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:11:57.067768 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.067626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x" event={"ID":"83f8e21b-264a-4a6f-a823-b6878475b2cd","Type":"ContainerStarted","Data":"16b39f81b7fe0a81354130cf09a357e0ab062f15d806600093373393d8468317"}
Sep 18 20:11:57.098265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.098209    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x" podStartSLOduration=1.837807212 podStartE2EDuration="2.098195083s" totalImagesPullingTime="260.387871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:57.096668375 +0000 UTC m=+897.657251826" watchObservedRunningTime="2026-09-18 20:11:57.098195083 +0000 UTC m=+897.658778539"
Sep 18 20:11:57.165563 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.165522    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phzl9\" (UniqueName: \"kubernetes.io/projected/9353cf66-a8b0-4281-86b1-90d2b12dc37a-kube-api-access-phzl9\") pod \"payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s\" (UID: \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:11:57.165764 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.165734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9353cf66-a8b0-4281-86b1-90d2b12dc37a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s\" (UID: \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:11:57.267401 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.267358    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9353cf66-a8b0-4281-86b1-90d2b12dc37a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s\" (UID: \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:11:57.267622 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.267458    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phzl9\" (UniqueName: \"kubernetes.io/projected/9353cf66-a8b0-4281-86b1-90d2b12dc37a-kube-api-access-phzl9\") pod \"payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s\" (UID: \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:11:57.268266 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.268237    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9353cf66-a8b0-4281-86b1-90d2b12dc37a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s\" (UID: \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:11:57.276301 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.276270    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phzl9\" (UniqueName: \"kubernetes.io/projected/9353cf66-a8b0-4281-86b1-90d2b12dc37a-kube-api-access-phzl9\") pod \"payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s\" (UID: \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:11:57.345928 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.345886    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:11:57.629603 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:57.629574    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"]
Sep 18 20:11:57.631601 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:11:57.631571    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9353cf66_a8b0_4281_86b1_90d2b12dc37a.slice/crio-8e06958c064d6e563600c12364ba1b96234fbe37646f296e5ec3821f837fab61 WatchSource:0}: Error finding container 8e06958c064d6e563600c12364ba1b96234fbe37646f296e5ec3821f837fab61: Status 404 returned error can't find the container with id 8e06958c064d6e563600c12364ba1b96234fbe37646f296e5ec3821f837fab61
Sep 18 20:11:58.074352 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:58.074246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s" event={"ID":"9353cf66-a8b0-4281-86b1-90d2b12dc37a","Type":"ContainerStarted","Data":"dcfcc425efd384fdc6e83ad7efb47b3265e7b6213dd1761e231152a036b3dec5"}
Sep 18 20:11:58.074352 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:58.074305    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s" event={"ID":"9353cf66-a8b0-4281-86b1-90d2b12dc37a","Type":"ContainerStarted","Data":"8e06958c064d6e563600c12364ba1b96234fbe37646f296e5ec3821f837fab61"}
Sep 18 20:11:58.115146 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:58.115056    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s" podStartSLOduration=1.115036222 podStartE2EDuration="1.115036222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:58.114507207 +0000 UTC m=+898.675090684" watchObservedRunningTime="2026-09-18 20:11:58.115036222 +0000 UTC m=+898.675619679"
Sep 18 20:11:59.080288 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:11:59.080256    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:12:01.685038 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:01.685003    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:12:01.686899 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:01.686875    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:12:02.040383 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:02.040303    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:12:03.080532 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:03.080499    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:12:10.089049 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:10.089014    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:12:22.803669 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:22.803627    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"]
Sep 18 20:12:22.804223 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:22.803913    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs" podUID="b7b3722a-3622-4a36-91ca-7224ca51eae8" containerName="payload-pre-processing" containerID="cri-o://75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4" gracePeriod=30
Sep 18 20:12:23.069522 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.069450    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:12:23.131663 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.131627    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b3722a-3622-4a36-91ca-7224ca51eae8-plugins-config-volume\") pod \"b7b3722a-3622-4a36-91ca-7224ca51eae8\" (UID: \"b7b3722a-3622-4a36-91ca-7224ca51eae8\") "
Sep 18 20:12:23.131878 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.131703    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnd4x\" (UniqueName: \"kubernetes.io/projected/b7b3722a-3622-4a36-91ca-7224ca51eae8-kube-api-access-jnd4x\") pod \"b7b3722a-3622-4a36-91ca-7224ca51eae8\" (UID: \"b7b3722a-3622-4a36-91ca-7224ca51eae8\") "
Sep 18 20:12:23.132156 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.132127    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7b3722a-3622-4a36-91ca-7224ca51eae8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7b3722a-3622-4a36-91ca-7224ca51eae8" (UID: "b7b3722a-3622-4a36-91ca-7224ca51eae8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:12:23.134268 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.134239    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7b3722a-3622-4a36-91ca-7224ca51eae8-kube-api-access-jnd4x" (OuterVolumeSpecName: "kube-api-access-jnd4x") pod "b7b3722a-3622-4a36-91ca-7224ca51eae8" (UID: "b7b3722a-3622-4a36-91ca-7224ca51eae8"). InnerVolumeSpecName "kube-api-access-jnd4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:12:23.222990 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.222947    2567 generic.go:356] "Generic (PLEG): container finished" podID="b7b3722a-3622-4a36-91ca-7224ca51eae8" containerID="75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4" exitCode=0
Sep 18 20:12:23.223170 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.223017    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"
Sep 18 20:12:23.223170 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.223034    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs" event={"ID":"b7b3722a-3622-4a36-91ca-7224ca51eae8","Type":"ContainerDied","Data":"75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4"}
Sep 18 20:12:23.223170 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.223112    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs" event={"ID":"b7b3722a-3622-4a36-91ca-7224ca51eae8","Type":"ContainerDied","Data":"370f1af4467ddfcf69ba08367e9fb26ebd78a4ba4e3657291fbbe5a0d94e6c01"}
Sep 18 20:12:23.223170 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.223142    2567 scope.go:117] "RemoveContainer" containerID="75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4"
Sep 18 20:12:23.232769 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.232742    2567 scope.go:117] "RemoveContainer" containerID="75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4"
Sep 18 20:12:23.233136 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.233113    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b3722a-3622-4a36-91ca-7224ca51eae8-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:12:23.233214 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.233145    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnd4x\" (UniqueName: \"kubernetes.io/projected/b7b3722a-3622-4a36-91ca-7224ca51eae8-kube-api-access-jnd4x\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:12:23.233214 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:12:23.233113    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4\": container with ID starting with 75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4 not found: ID does not exist" containerID="75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4"
Sep 18 20:12:23.233214 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.233174    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4"} err="failed to get container status \"75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4\": rpc error: code = NotFound desc = could not find container \"75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4\": container with ID starting with 75bea681300a855cc24c0bcf1cb66116f142e56c47b9ac005ff004a5f3e52de4 not found: ID does not exist"
Sep 18 20:12:23.256554 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.256518    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"]
Sep 18 20:12:23.266443 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.266405    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-0a7e3fae-7cdb84fc7d-7qqgs"]
Sep 18 20:12:23.700023 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:23.699987    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7b3722a-3622-4a36-91ca-7224ca51eae8" path="/var/lib/kubelet/pods/b7b3722a-3622-4a36-91ca-7224ca51eae8/volumes"
Sep 18 20:12:31.899203 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:31.899171    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"]
Sep 18 20:12:31.899843 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:31.899827    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7b3722a-3622-4a36-91ca-7224ca51eae8" containerName="payload-pre-processing"
Sep 18 20:12:31.899908 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:31.899845    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7b3722a-3622-4a36-91ca-7224ca51eae8" containerName="payload-pre-processing"
Sep 18 20:12:31.899984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:31.899972    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7b3722a-3622-4a36-91ca-7224ca51eae8" containerName="payload-pre-processing"
Sep 18 20:12:31.904999 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:31.904975    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:12:31.908015 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:31.907993    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-7bed33dc\""
Sep 18 20:12:31.908161 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:31.908122    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-7bed33dc-dockercfg-b98cr\""
Sep 18 20:12:31.920212 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:31.920179    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"]
Sep 18 20:12:32.030291 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:32.030253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkfmt\" (UniqueName: \"kubernetes.io/projected/18f9acc9-6f44-40e2-808a-84babaf91e74-kube-api-access-jkfmt\") pod \"payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj\" (UID: \"18f9acc9-6f44-40e2-808a-84babaf91e74\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:12:32.030495 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:32.030418    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f9acc9-6f44-40e2-808a-84babaf91e74-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj\" (UID: \"18f9acc9-6f44-40e2-808a-84babaf91e74\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:12:32.132043 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:32.131990    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkfmt\" (UniqueName: \"kubernetes.io/projected/18f9acc9-6f44-40e2-808a-84babaf91e74-kube-api-access-jkfmt\") pod \"payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj\" (UID: \"18f9acc9-6f44-40e2-808a-84babaf91e74\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:12:32.132270 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:32.132157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f9acc9-6f44-40e2-808a-84babaf91e74-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj\" (UID: \"18f9acc9-6f44-40e2-808a-84babaf91e74\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:12:32.132921 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:32.132900    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f9acc9-6f44-40e2-808a-84babaf91e74-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj\" (UID: \"18f9acc9-6f44-40e2-808a-84babaf91e74\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:12:32.143133 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:32.143102    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkfmt\" (UniqueName: \"kubernetes.io/projected/18f9acc9-6f44-40e2-808a-84babaf91e74-kube-api-access-jkfmt\") pod \"payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj\" (UID: \"18f9acc9-6f44-40e2-808a-84babaf91e74\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:12:32.218847 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:32.218753    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:12:32.499160 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:32.499129    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"]
Sep 18 20:12:32.500029 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:12:32.499999    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18f9acc9_6f44_40e2_808a_84babaf91e74.slice/crio-1e5b460237947e41f9afa17c9cab1c90a50ec1a541b456dc865c48d7a1db5cb3 WatchSource:0}: Error finding container 1e5b460237947e41f9afa17c9cab1c90a50ec1a541b456dc865c48d7a1db5cb3: Status 404 returned error can't find the container with id 1e5b460237947e41f9afa17c9cab1c90a50ec1a541b456dc865c48d7a1db5cb3
Sep 18 20:12:33.280574 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:33.280533    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj" event={"ID":"18f9acc9-6f44-40e2-808a-84babaf91e74","Type":"ContainerStarted","Data":"1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb"}
Sep 18 20:12:33.281006 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:33.280607    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:12:33.281006 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:33.280628    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj" event={"ID":"18f9acc9-6f44-40e2-808a-84babaf91e74","Type":"ContainerStarted","Data":"1e5b460237947e41f9afa17c9cab1c90a50ec1a541b456dc865c48d7a1db5cb3"}
Sep 18 20:12:33.307308 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:33.307245    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj" podStartSLOduration=2.307225491 podStartE2EDuration="2.307225491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:12:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:12:33.305015257 +0000 UTC m=+933.865598714" watchObservedRunningTime="2026-09-18 20:12:33.307225491 +0000 UTC m=+933.867808979"
Sep 18 20:12:44.288486 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:12:44.288442    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:13:08.853517 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:08.853463    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"]
Sep 18 20:13:08.854165 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:08.853729    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj" podUID="18f9acc9-6f44-40e2-808a-84babaf91e74" containerName="payload-pre-processing" containerID="cri-o://1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb" gracePeriod=30
Sep 18 20:13:09.128513 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.128485    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:13:09.220151 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.220116    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jkfmt\" (UniqueName: \"kubernetes.io/projected/18f9acc9-6f44-40e2-808a-84babaf91e74-kube-api-access-jkfmt\") pod \"18f9acc9-6f44-40e2-808a-84babaf91e74\" (UID: \"18f9acc9-6f44-40e2-808a-84babaf91e74\") "
Sep 18 20:13:09.220359 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.220336    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f9acc9-6f44-40e2-808a-84babaf91e74-plugins-config-volume\") pod \"18f9acc9-6f44-40e2-808a-84babaf91e74\" (UID: \"18f9acc9-6f44-40e2-808a-84babaf91e74\") "
Sep 18 20:13:09.220679 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.220647    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18f9acc9-6f44-40e2-808a-84babaf91e74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "18f9acc9-6f44-40e2-808a-84babaf91e74" (UID: "18f9acc9-6f44-40e2-808a-84babaf91e74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:13:09.222206 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.222181    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18f9acc9-6f44-40e2-808a-84babaf91e74-kube-api-access-jkfmt" (OuterVolumeSpecName: "kube-api-access-jkfmt") pod "18f9acc9-6f44-40e2-808a-84babaf91e74" (UID: "18f9acc9-6f44-40e2-808a-84babaf91e74"). InnerVolumeSpecName "kube-api-access-jkfmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:13:09.321888 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.321852    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/18f9acc9-6f44-40e2-808a-84babaf91e74-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:13:09.321888 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.321885    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkfmt\" (UniqueName: \"kubernetes.io/projected/18f9acc9-6f44-40e2-808a-84babaf91e74-kube-api-access-jkfmt\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:13:09.484722 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.484607    2567 generic.go:356] "Generic (PLEG): container finished" podID="18f9acc9-6f44-40e2-808a-84babaf91e74" containerID="1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb" exitCode=0
Sep 18 20:13:09.484722 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.484648    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"
Sep 18 20:13:09.484975 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.484715    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj" event={"ID":"18f9acc9-6f44-40e2-808a-84babaf91e74","Type":"ContainerDied","Data":"1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb"}
Sep 18 20:13:09.484975 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.484768    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj" event={"ID":"18f9acc9-6f44-40e2-808a-84babaf91e74","Type":"ContainerDied","Data":"1e5b460237947e41f9afa17c9cab1c90a50ec1a541b456dc865c48d7a1db5cb3"}
Sep 18 20:13:09.484975 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.484797    2567 scope.go:117] "RemoveContainer" containerID="1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb"
Sep 18 20:13:09.494791 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.494766    2567 scope.go:117] "RemoveContainer" containerID="1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb"
Sep 18 20:13:09.495119 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:13:09.495097    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb\": container with ID starting with 1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb not found: ID does not exist" containerID="1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb"
Sep 18 20:13:09.495172 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.495128    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb"} err="failed to get container status \"1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb\": rpc error: code = NotFound desc = could not find container \"1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb\": container with ID starting with 1a4df883c84e4530a347c59dbf1d767822cd3ee366acac16037ff14aaab597bb not found: ID does not exist"
Sep 18 20:13:09.511170 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.511132    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"]
Sep 18 20:13:09.514267 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.514238    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-7bed33dc-76c564f54-4vlgj"]
Sep 18 20:13:09.700623 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:09.700584    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18f9acc9-6f44-40e2-808a-84babaf91e74" path="/var/lib/kubelet/pods/18f9acc9-6f44-40e2-808a-84babaf91e74/volumes"
Sep 18 20:13:16.303785 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.303706    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"]
Sep 18 20:13:16.304232 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.304216    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18f9acc9-6f44-40e2-808a-84babaf91e74" containerName="payload-pre-processing"
Sep 18 20:13:16.304232 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.304233    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="18f9acc9-6f44-40e2-808a-84babaf91e74" containerName="payload-pre-processing"
Sep 18 20:13:16.304314 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.304302    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="18f9acc9-6f44-40e2-808a-84babaf91e74" containerName="payload-pre-processing"
Sep 18 20:13:16.309134 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.309114    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:16.311627 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.311597    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f524d4ec-dockercfg-jm6tl\""
Sep 18 20:13:16.311792 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.311610    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f524d4ec\""
Sep 18 20:13:16.319247 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.319220    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"]
Sep 18 20:13:16.392546 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.392506    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292\" (UID: \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:16.392773 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.392571    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrrtl\" (UniqueName: \"kubernetes.io/projected/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-kube-api-access-vrrtl\") pod \"payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292\" (UID: \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:16.493379 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.493330    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292\" (UID: \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:16.493574 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.493467    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrrtl\" (UniqueName: \"kubernetes.io/projected/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-kube-api-access-vrrtl\") pod \"payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292\" (UID: \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:16.494060 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.494035    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292\" (UID: \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:16.502065 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.502036    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrrtl\" (UniqueName: \"kubernetes.io/projected/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-kube-api-access-vrrtl\") pod \"payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292\" (UID: \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:16.621752 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.621711    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:16.896457 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:16.896422    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"]
Sep 18 20:13:16.898519 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:13:16.898488    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dfea432_d08a_4156_9154_fd9f9e3dc9a8.slice/crio-dda7e4ba727feeb726a9b8a6d3bf8b3137270d475c1a868ebd15393c798206a1 WatchSource:0}: Error finding container dda7e4ba727feeb726a9b8a6d3bf8b3137270d475c1a868ebd15393c798206a1: Status 404 returned error can't find the container with id dda7e4ba727feeb726a9b8a6d3bf8b3137270d475c1a868ebd15393c798206a1
Sep 18 20:13:17.541660 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:17.541624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292" event={"ID":"2dfea432-d08a-4156-9154-fd9f9e3dc9a8","Type":"ContainerStarted","Data":"da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36"}
Sep 18 20:13:17.542106 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:17.541699    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292" event={"ID":"2dfea432-d08a-4156-9154-fd9f9e3dc9a8","Type":"ContainerStarted","Data":"dda7e4ba727feeb726a9b8a6d3bf8b3137270d475c1a868ebd15393c798206a1"}
Sep 18 20:13:17.542106 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:17.541803    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:17.566173 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:17.566112    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292" podStartSLOduration=1.566089711 podStartE2EDuration="1.566089711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:13:17.565408295 +0000 UTC m=+978.125991751" watchObservedRunningTime="2026-09-18 20:13:17.566089711 +0000 UTC m=+978.126673171"
Sep 18 20:13:28.550523 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:28.550492    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:54.110303 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.110268    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"]
Sep 18 20:13:54.114148 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.114129    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.117175 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.117153    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-d0d77c\"/\"kube-root-ca.crt\""
Sep 18 20:13:54.117620 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.117604    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-d0d77c\"/\"default-dockercfg-b9gzc\""
Sep 18 20:13:54.117672 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.117625    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-d0d77c\"/\"autores-model-d0d77c-kserve-self-signed-certs\""
Sep 18 20:13:54.117672 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.117648    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-d0d77c\"/\"openshift-service-ca.crt\""
Sep 18 20:13:54.125835 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.125804    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"]
Sep 18 20:13:54.193134 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.193096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-dshm\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.193134 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.193136    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-home\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.193344 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.193194    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tls-certs\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.193344 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.193282    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c792\" (UniqueName: \"kubernetes.io/projected/03a2b8f0-11c5-45c9-9107-ff8345de00c2-kube-api-access-2c792\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.193344 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.193332    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-model-cache\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.193458 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.193366    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tmp-dir\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.294781 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.294738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tls-certs\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.294984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.294843    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c792\" (UniqueName: \"kubernetes.io/projected/03a2b8f0-11c5-45c9-9107-ff8345de00c2-kube-api-access-2c792\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.294984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.294887    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-model-cache\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.295109 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.295067    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tmp-dir\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.295193 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.295177    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-dshm\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.295252 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.295212    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-home\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.295310 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.295294    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-model-cache\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.295450 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.295433    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tmp-dir\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.295570 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.295546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-home\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.297276 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.297255    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-dshm\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.297558 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.297539    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tls-certs\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.303228 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.303205    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c792\" (UniqueName: \"kubernetes.io/projected/03a2b8f0-11c5-45c9-9107-ff8345de00c2-kube-api-access-2c792\") pod \"autores-model-d0d77c-kserve-8555d8dbbc-mxbc8\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") " pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.424978 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.424877    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:13:54.702732 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.702704    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"]
Sep 18 20:13:54.704531 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:13:54.704497    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03a2b8f0_11c5_45c9_9107_ff8345de00c2.slice/crio-14ff9281a5016ffa1f8b54dad21d1a2bfb142c460e5d4359ca8e0acedc402e16 WatchSource:0}: Error finding container 14ff9281a5016ffa1f8b54dad21d1a2bfb142c460e5d4359ca8e0acedc402e16: Status 404 returned error can't find the container with id 14ff9281a5016ffa1f8b54dad21d1a2bfb142c460e5d4359ca8e0acedc402e16
Sep 18 20:13:54.752274 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:54.752245    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8" event={"ID":"03a2b8f0-11c5-45c9-9107-ff8345de00c2","Type":"ContainerStarted","Data":"14ff9281a5016ffa1f8b54dad21d1a2bfb142c460e5d4359ca8e0acedc402e16"}
Sep 18 20:13:55.140913 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.140874    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"]
Sep 18 20:13:55.141378 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.141172    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292" podUID="2dfea432-d08a-4156-9154-fd9f9e3dc9a8" containerName="payload-pre-processing" containerID="cri-o://da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36" gracePeriod=30
Sep 18 20:13:55.408833 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.408757    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:55.507852 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.507816    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrrtl\" (UniqueName: \"kubernetes.io/projected/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-kube-api-access-vrrtl\") pod \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\" (UID: \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\") "
Sep 18 20:13:55.508086 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.508069    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-plugins-config-volume\") pod \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\" (UID: \"2dfea432-d08a-4156-9154-fd9f9e3dc9a8\") "
Sep 18 20:13:55.508420 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.508396    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2dfea432-d08a-4156-9154-fd9f9e3dc9a8" (UID: "2dfea432-d08a-4156-9154-fd9f9e3dc9a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:13:55.509816 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.509788    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-kube-api-access-vrrtl" (OuterVolumeSpecName: "kube-api-access-vrrtl") pod "2dfea432-d08a-4156-9154-fd9f9e3dc9a8" (UID: "2dfea432-d08a-4156-9154-fd9f9e3dc9a8"). InnerVolumeSpecName "kube-api-access-vrrtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:13:55.609015 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.608976    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:13:55.609015 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.609009    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrrtl\" (UniqueName: \"kubernetes.io/projected/2dfea432-d08a-4156-9154-fd9f9e3dc9a8-kube-api-access-vrrtl\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:13:55.760364 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.760327    2567 generic.go:356] "Generic (PLEG): container finished" podID="2dfea432-d08a-4156-9154-fd9f9e3dc9a8" containerID="da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36" exitCode=0
Sep 18 20:13:55.760582 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.760372    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"
Sep 18 20:13:55.760582 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.760407    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292" event={"ID":"2dfea432-d08a-4156-9154-fd9f9e3dc9a8","Type":"ContainerDied","Data":"da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36"}
Sep 18 20:13:55.760582 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.760460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292" event={"ID":"2dfea432-d08a-4156-9154-fd9f9e3dc9a8","Type":"ContainerDied","Data":"dda7e4ba727feeb726a9b8a6d3bf8b3137270d475c1a868ebd15393c798206a1"}
Sep 18 20:13:55.760582 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.760484    2567 scope.go:117] "RemoveContainer" containerID="da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36"
Sep 18 20:13:55.767193 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.767107    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8" event={"ID":"03a2b8f0-11c5-45c9-9107-ff8345de00c2","Type":"ContainerStarted","Data":"e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c"}
Sep 18 20:13:55.773972 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.773953    2567 scope.go:117] "RemoveContainer" containerID="da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36"
Sep 18 20:13:55.774317 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:13:55.774297    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36\": container with ID starting with da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36 not found: ID does not exist" containerID="da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36"
Sep 18 20:13:55.774379 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.774330    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36"} err="failed to get container status \"da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36\": rpc error: code = NotFound desc = could not find container \"da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36\": container with ID starting with da6a708b273e9452869c7ed5ee30ddd9ad20f9cb3d5ead71bf24fe8f4ebb9b36 not found: ID does not exist"
Sep 18 20:13:55.791415 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.791353    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8" podStartSLOduration=1.791334964 podStartE2EDuration="1.791334964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:13:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:13:55.789416192 +0000 UTC m=+1016.349999650" watchObservedRunningTime="2026-09-18 20:13:55.791334964 +0000 UTC m=+1016.351918422"
Sep 18 20:13:55.806015 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.805985    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"]
Sep 18 20:13:55.808928 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:55.808903    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f524d4ec-7d9cbb5cb6-tp292"]
Sep 18 20:13:57.701437 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:13:57.701399    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dfea432-d08a-4156-9154-fd9f9e3dc9a8" path="/var/lib/kubelet/pods/2dfea432-d08a-4156-9154-fd9f9e3dc9a8/volumes"
Sep 18 20:14:04.425982 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:04.425935    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:14:04.426406 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:04.426037    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:14:04.439132 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:04.439102    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:14:04.837195 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:04.837106    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:14:20.806901 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.806855    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"]
Sep 18 20:14:20.807605 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.807583    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dfea432-d08a-4156-9154-fd9f9e3dc9a8" containerName="payload-pre-processing"
Sep 18 20:14:20.807663 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.807609    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dfea432-d08a-4156-9154-fd9f9e3dc9a8" containerName="payload-pre-processing"
Sep 18 20:14:20.807794 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.807777    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dfea432-d08a-4156-9154-fd9f9e3dc9a8" containerName="payload-pre-processing"
Sep 18 20:14:20.811536 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.811514    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.814053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.814029    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3a8052\"/\"openshift-service-ca.crt\""
Sep 18 20:14:20.814770 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.814749    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3a8052\"/\"default-dockercfg-thbpb\""
Sep 18 20:14:20.814873 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.814776    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3a8052\"/\"kube-root-ca.crt\""
Sep 18 20:14:20.814873 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.814776    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3a8052\"/\"test-model-3a8052-kserve-self-signed-certs\""
Sep 18 20:14:20.822785 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.822761    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"]
Sep 18 20:14:20.869570 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.869522    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cg46\" (UniqueName: \"kubernetes.io/projected/46b59d31-a96b-4993-ba45-80b8d7b89fae-kube-api-access-6cg46\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.869570 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.869574    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-tmp-dir\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.869830 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.869650    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-model-cache\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.869830 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.869769    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-dshm\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.869830 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.869808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-home\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.869830 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.869824    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46b59d31-a96b-4993-ba45-80b8d7b89fae-tls-certs\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.970405 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.970361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cg46\" (UniqueName: \"kubernetes.io/projected/46b59d31-a96b-4993-ba45-80b8d7b89fae-kube-api-access-6cg46\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.970640 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.970429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-tmp-dir\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.970640 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.970514    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-model-cache\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.970640 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.970584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-dshm\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.970640 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.970630    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-home\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.970912 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.970652    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46b59d31-a96b-4993-ba45-80b8d7b89fae-tls-certs\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.971025 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.970999    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-tmp-dir\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.971448 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.971424    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-model-cache\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.971569 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.971550    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-home\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.973185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.973159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-dshm\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.973357 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.973342    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46b59d31-a96b-4993-ba45-80b8d7b89fae-tls-certs\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:20.980547 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:20.980524    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cg46\" (UniqueName: \"kubernetes.io/projected/46b59d31-a96b-4993-ba45-80b8d7b89fae-kube-api-access-6cg46\") pod \"test-model-3a8052-kserve-867c6b6cdc-4vjrn\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") " pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:21.124410 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:21.124375    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:21.399650 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:21.399615    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"]
Sep 18 20:14:21.400926 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:14:21.400898    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46b59d31_a96b_4993_ba45_80b8d7b89fae.slice/crio-56fff35fbef5640df655fb78840352afb6087e0d7e7e44322c9982622967afed WatchSource:0}: Error finding container 56fff35fbef5640df655fb78840352afb6087e0d7e7e44322c9982622967afed: Status 404 returned error can't find the container with id 56fff35fbef5640df655fb78840352afb6087e0d7e7e44322c9982622967afed
Sep 18 20:14:21.922662 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:21.922629    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn" event={"ID":"46b59d31-a96b-4993-ba45-80b8d7b89fae","Type":"ContainerStarted","Data":"5d6e0433b5bce6255b6a0796da2d913a344e6dbc5f4d27fbc61d3d070d6526d3"}
Sep 18 20:14:21.923071 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:21.922672    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn" event={"ID":"46b59d31-a96b-4993-ba45-80b8d7b89fae","Type":"ContainerStarted","Data":"56fff35fbef5640df655fb78840352afb6087e0d7e7e44322c9982622967afed"}
Sep 18 20:14:31.124569 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:31.124530    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:31.124990 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:31.124854    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:31.137348 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:31.137317    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:31.164352 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:31.164293    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn" podStartSLOduration=11.164276097 podStartE2EDuration="11.164276097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:14:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:14:21.956099405 +0000 UTC m=+1042.516682863" watchObservedRunningTime="2026-09-18 20:14:31.164276097 +0000 UTC m=+1051.724859756"
Sep 18 20:14:31.990248 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:31.990213    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:14:37.677836 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:37.677790    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"]
Sep 18 20:14:37.678320 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:37.678278    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv" podUID="28eb230e-55ef-44e2-9676-7eb850504065" containerName="maas-api" containerID="cri-o://65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb" gracePeriod=30
Sep 18 20:14:37.934461 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:37.934389    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:14:38.011830 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.011795    2567 generic.go:356] "Generic (PLEG): container finished" podID="28eb230e-55ef-44e2-9676-7eb850504065" containerID="65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb" exitCode=0
Sep 18 20:14:38.011830 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.011842    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"
Sep 18 20:14:38.012042 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.011854    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv" event={"ID":"28eb230e-55ef-44e2-9676-7eb850504065","Type":"ContainerDied","Data":"65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb"}
Sep 18 20:14:38.012042 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.011900    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv" event={"ID":"28eb230e-55ef-44e2-9676-7eb850504065","Type":"ContainerDied","Data":"a28ed816d2715ef198cf1e587506dabd4c26fa688c0367e03d834e75c3315da0"}
Sep 18 20:14:38.012042 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.011920    2567 scope.go:117] "RemoveContainer" containerID="65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb"
Sep 18 20:14:38.021906 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.021882    2567 scope.go:117] "RemoveContainer" containerID="65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb"
Sep 18 20:14:38.022302 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:14:38.022274    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb\": container with ID starting with 65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb not found: ID does not exist" containerID="65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb"
Sep 18 20:14:38.022441 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.022307    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb"} err="failed to get container status \"65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb\": rpc error: code = NotFound desc = could not find container \"65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb\": container with ID starting with 65823cf1d9f188e57eb1f2923f870a3f7591f4f69ed76b7ef0cfa170efb778eb not found: ID does not exist"
Sep 18 20:14:38.053548 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.053507    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-metrics-certs\") pod \"28eb230e-55ef-44e2-9676-7eb850504065\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") "
Sep 18 20:14:38.053852 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.053570    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28eb230e-55ef-44e2-9676-7eb850504065-openshift-service-ca\") pod \"28eb230e-55ef-44e2-9676-7eb850504065\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") "
Sep 18 20:14:38.053852 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.053678    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-maas-api-tls\") pod \"28eb230e-55ef-44e2-9676-7eb850504065\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") "
Sep 18 20:14:38.053852 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.053817    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2nxrx\" (UniqueName: \"kubernetes.io/projected/28eb230e-55ef-44e2-9676-7eb850504065-kube-api-access-2nxrx\") pod \"28eb230e-55ef-44e2-9676-7eb850504065\" (UID: \"28eb230e-55ef-44e2-9676-7eb850504065\") "
Sep 18 20:14:38.054059 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.053995    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28eb230e-55ef-44e2-9676-7eb850504065-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28eb230e-55ef-44e2-9676-7eb850504065" (UID: "28eb230e-55ef-44e2-9676-7eb850504065"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:14:38.054427 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.054269    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28eb230e-55ef-44e2-9676-7eb850504065-openshift-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:38.056509 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.056467    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "28eb230e-55ef-44e2-9676-7eb850504065" (UID: "28eb230e-55ef-44e2-9676-7eb850504065"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:14:38.057214 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.057106    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28eb230e-55ef-44e2-9676-7eb850504065-kube-api-access-2nxrx" (OuterVolumeSpecName: "kube-api-access-2nxrx") pod "28eb230e-55ef-44e2-9676-7eb850504065" (UID: "28eb230e-55ef-44e2-9676-7eb850504065"). InnerVolumeSpecName "kube-api-access-2nxrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:14:38.057403 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.057374    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28eb230e-55ef-44e2-9676-7eb850504065" (UID: "28eb230e-55ef-44e2-9676-7eb850504065"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:14:38.155282 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.155234    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:38.155282 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.155268    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28eb230e-55ef-44e2-9676-7eb850504065-maas-api-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:38.155282 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.155283    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nxrx\" (UniqueName: \"kubernetes.io/projected/28eb230e-55ef-44e2-9676-7eb850504065-kube-api-access-2nxrx\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:38.339469 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.339435    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"]
Sep 18 20:14:38.344606 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:38.344570    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-d0d77c-6cb888fc9c-5v9tv"]
Sep 18 20:14:39.699894 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:39.699858    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28eb230e-55ef-44e2-9676-7eb850504065" path="/var/lib/kubelet/pods/28eb230e-55ef-44e2-9676-7eb850504065/volumes"
Sep 18 20:14:43.430077 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:43.429995    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"]
Sep 18 20:14:58.451138 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.451076    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8" podUID="03a2b8f0-11c5-45c9-9107-ff8345de00c2" containerName="main" containerID="cri-o://e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c" gracePeriod=45
Sep 18 20:14:58.704073 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.704009    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:14:58.786380 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.786324    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tmp-dir\") pod \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") "
Sep 18 20:14:58.786589 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.786478    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-model-cache\") pod \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") "
Sep 18 20:14:58.786589 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.786506    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-dshm\") pod \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") "
Sep 18 20:14:58.786589 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.786528    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tls-certs\") pod \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") "
Sep 18 20:14:58.786589 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.786566    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-home\") pod \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") "
Sep 18 20:14:58.786844 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.786600    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c792\" (UniqueName: \"kubernetes.io/projected/03a2b8f0-11c5-45c9-9107-ff8345de00c2-kube-api-access-2c792\") pod \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\" (UID: \"03a2b8f0-11c5-45c9-9107-ff8345de00c2\") "
Sep 18 20:14:58.786844 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.786627    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "03a2b8f0-11c5-45c9-9107-ff8345de00c2" (UID: "03a2b8f0-11c5-45c9-9107-ff8345de00c2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:14:58.786844 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.786776    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-model-cache" (OuterVolumeSpecName: "model-cache") pod "03a2b8f0-11c5-45c9-9107-ff8345de00c2" (UID: "03a2b8f0-11c5-45c9-9107-ff8345de00c2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:14:58.786992 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.786907    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-home" (OuterVolumeSpecName: "home") pod "03a2b8f0-11c5-45c9-9107-ff8345de00c2" (UID: "03a2b8f0-11c5-45c9-9107-ff8345de00c2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:14:58.787049 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.787038    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-home\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:58.787102 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.787057    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tmp-dir\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:58.787102 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.787073    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-model-cache\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:58.788918 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.788890    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-dshm" (OuterVolumeSpecName: "dshm") pod "03a2b8f0-11c5-45c9-9107-ff8345de00c2" (UID: "03a2b8f0-11c5-45c9-9107-ff8345de00c2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:14:58.789033 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.788941    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "03a2b8f0-11c5-45c9-9107-ff8345de00c2" (UID: "03a2b8f0-11c5-45c9-9107-ff8345de00c2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:14:58.789033 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.788954    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03a2b8f0-11c5-45c9-9107-ff8345de00c2-kube-api-access-2c792" (OuterVolumeSpecName: "kube-api-access-2c792") pod "03a2b8f0-11c5-45c9-9107-ff8345de00c2" (UID: "03a2b8f0-11c5-45c9-9107-ff8345de00c2"). InnerVolumeSpecName "kube-api-access-2c792". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:14:58.887860 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.887806    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03a2b8f0-11c5-45c9-9107-ff8345de00c2-dshm\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:58.887860 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.887851    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03a2b8f0-11c5-45c9-9107-ff8345de00c2-tls-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:58.887860 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:58.887864    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c792\" (UniqueName: \"kubernetes.io/projected/03a2b8f0-11c5-45c9-9107-ff8345de00c2-kube-api-access-2c792\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:14:59.134371 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.134335    2567 generic.go:356] "Generic (PLEG): container finished" podID="03a2b8f0-11c5-45c9-9107-ff8345de00c2" containerID="e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c" exitCode=0
Sep 18 20:14:59.134571 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.134378    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"
Sep 18 20:14:59.134571 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.134424    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8" event={"ID":"03a2b8f0-11c5-45c9-9107-ff8345de00c2","Type":"ContainerDied","Data":"e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c"}
Sep 18 20:14:59.134571 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.134466    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8" event={"ID":"03a2b8f0-11c5-45c9-9107-ff8345de00c2","Type":"ContainerDied","Data":"14ff9281a5016ffa1f8b54dad21d1a2bfb142c460e5d4359ca8e0acedc402e16"}
Sep 18 20:14:59.134571 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.134487    2567 scope.go:117] "RemoveContainer" containerID="e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c"
Sep 18 20:14:59.144579 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.144557    2567 scope.go:117] "RemoveContainer" containerID="e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c"
Sep 18 20:14:59.144935 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:14:59.144911    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c\": container with ID starting with e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c not found: ID does not exist" containerID="e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c"
Sep 18 20:14:59.145004 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.144952    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c"} err="failed to get container status \"e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c\": rpc error: code = NotFound desc = could not find container \"e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c\": container with ID starting with e93322599decc60c27197cd8b0af7fbec456899ced09e097c5039e3b83aa874c not found: ID does not exist"
Sep 18 20:14:59.160522 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.160490    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"]
Sep 18 20:14:59.173542 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.173505    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-d0d77c/autores-model-d0d77c-kserve-8555d8dbbc-mxbc8"]
Sep 18 20:14:59.665785 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.665754    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"]
Sep 18 20:14:59.666250 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.665984    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj" podUID="620af298-7d5b-4723-9365-12e31947e124" containerName="maas-api" containerID="cri-o://e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c" gracePeriod=30
Sep 18 20:14:59.700628 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.700595    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03a2b8f0-11c5-45c9-9107-ff8345de00c2" path="/var/lib/kubelet/pods/03a2b8f0-11c5-45c9-9107-ff8345de00c2/volumes"
Sep 18 20:14:59.730563 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.730527    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"]
Sep 18 20:14:59.730804 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.730774    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq" podUID="e21e30ad-89e2-474d-8614-6b330628eecb" containerName="payload-processing" containerID="cri-o://5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde" gracePeriod=30
Sep 18 20:14:59.991314 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:14:59.991285    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:15:00.020855 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.020827    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:15:00.100244 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.100208    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg6ss\" (UniqueName: \"kubernetes.io/projected/e21e30ad-89e2-474d-8614-6b330628eecb-kube-api-access-tg6ss\") pod \"e21e30ad-89e2-474d-8614-6b330628eecb\" (UID: \"e21e30ad-89e2-474d-8614-6b330628eecb\") "
Sep 18 20:15:00.100442 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.100257    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q6z5\" (UniqueName: \"kubernetes.io/projected/620af298-7d5b-4723-9365-12e31947e124-kube-api-access-7q6z5\") pod \"620af298-7d5b-4723-9365-12e31947e124\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") "
Sep 18 20:15:00.100442 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.100277    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e21e30ad-89e2-474d-8614-6b330628eecb-plugins-config-volume\") pod \"e21e30ad-89e2-474d-8614-6b330628eecb\" (UID: \"e21e30ad-89e2-474d-8614-6b330628eecb\") "
Sep 18 20:15:00.100442 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.100359    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-maas-api-tls\") pod \"620af298-7d5b-4723-9365-12e31947e124\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") "
Sep 18 20:15:00.100442 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.100399    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-metrics-certs\") pod \"620af298-7d5b-4723-9365-12e31947e124\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") "
Sep 18 20:15:00.100665 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.100449    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/620af298-7d5b-4723-9365-12e31947e124-openshift-service-ca\") pod \"620af298-7d5b-4723-9365-12e31947e124\" (UID: \"620af298-7d5b-4723-9365-12e31947e124\") "
Sep 18 20:15:00.100757 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.100710    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e21e30ad-89e2-474d-8614-6b330628eecb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e21e30ad-89e2-474d-8614-6b330628eecb" (UID: "e21e30ad-89e2-474d-8614-6b330628eecb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:15:00.101099 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.101068    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/620af298-7d5b-4723-9365-12e31947e124-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "620af298-7d5b-4723-9365-12e31947e124" (UID: "620af298-7d5b-4723-9365-12e31947e124"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:15:00.101209 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.101141    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e21e30ad-89e2-474d-8614-6b330628eecb-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:15:00.102739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.102679    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "620af298-7d5b-4723-9365-12e31947e124" (UID: "620af298-7d5b-4723-9365-12e31947e124"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:15:00.102838 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.102710    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e21e30ad-89e2-474d-8614-6b330628eecb-kube-api-access-tg6ss" (OuterVolumeSpecName: "kube-api-access-tg6ss") pod "e21e30ad-89e2-474d-8614-6b330628eecb" (UID: "e21e30ad-89e2-474d-8614-6b330628eecb"). InnerVolumeSpecName "kube-api-access-tg6ss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:15:00.102838 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.102761    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/620af298-7d5b-4723-9365-12e31947e124-kube-api-access-7q6z5" (OuterVolumeSpecName: "kube-api-access-7q6z5") pod "620af298-7d5b-4723-9365-12e31947e124" (UID: "620af298-7d5b-4723-9365-12e31947e124"). InnerVolumeSpecName "kube-api-access-7q6z5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:15:00.102838 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.102779    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "620af298-7d5b-4723-9365-12e31947e124" (UID: "620af298-7d5b-4723-9365-12e31947e124"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:15:00.142845 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.142811    2567 generic.go:356] "Generic (PLEG): container finished" podID="e21e30ad-89e2-474d-8614-6b330628eecb" containerID="5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde" exitCode=0
Sep 18 20:15:00.143017 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.142856    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"
Sep 18 20:15:00.143017 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.142896    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq" event={"ID":"e21e30ad-89e2-474d-8614-6b330628eecb","Type":"ContainerDied","Data":"5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde"}
Sep 18 20:15:00.143017 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.142949    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq" event={"ID":"e21e30ad-89e2-474d-8614-6b330628eecb","Type":"ContainerDied","Data":"1bcbbdebeb73442e8b65ddbdc2335d02fa55095d39a38940b67dd2f3fe5e7fee"}
Sep 18 20:15:00.143017 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.142979    2567 scope.go:117] "RemoveContainer" containerID="5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde"
Sep 18 20:15:00.144328 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.144298    2567 generic.go:356] "Generic (PLEG): container finished" podID="620af298-7d5b-4723-9365-12e31947e124" containerID="e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c" exitCode=0
Sep 18 20:15:00.144424 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.144341    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"
Sep 18 20:15:00.144424 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.144344    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj" event={"ID":"620af298-7d5b-4723-9365-12e31947e124","Type":"ContainerDied","Data":"e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c"}
Sep 18 20:15:00.144424 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.144398    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj" event={"ID":"620af298-7d5b-4723-9365-12e31947e124","Type":"ContainerDied","Data":"09d2add378311500aa29649cadf912645aef88a1006594ab08517e4b1ef6656d"}
Sep 18 20:15:00.153550 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.153527    2567 scope.go:117] "RemoveContainer" containerID="5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde"
Sep 18 20:15:00.153892 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:15:00.153871    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde\": container with ID starting with 5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde not found: ID does not exist" containerID="5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde"
Sep 18 20:15:00.153964 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.153906    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde"} err="failed to get container status \"5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde\": rpc error: code = NotFound desc = could not find container \"5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde\": container with ID starting with 5f5e91905810a3a0262729c9979bd1a39d93bd3569106f6a89db64a9018e4bde not found: ID does not exist"
Sep 18 20:15:00.153964 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.153933    2567 scope.go:117] "RemoveContainer" containerID="e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c"
Sep 18 20:15:00.167058 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.167037    2567 scope.go:117] "RemoveContainer" containerID="e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c"
Sep 18 20:15:00.167412 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:15:00.167392    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c\": container with ID starting with e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c not found: ID does not exist" containerID="e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c"
Sep 18 20:15:00.167477 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.167420    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c"} err="failed to get container status \"e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c\": rpc error: code = NotFound desc = could not find container \"e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c\": container with ID starting with e3c117adb2ec35f6332b297feb40ec1a7c6e9f715c6f0c276991391c6a21e55c not found: ID does not exist"
Sep 18 20:15:00.194409 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.194322    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"]
Sep 18 20:15:00.200074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200041    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5"]
Sep 18 20:15:00.200609 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200590    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03a2b8f0-11c5-45c9-9107-ff8345de00c2" containerName="main"
Sep 18 20:15:00.200609 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200611    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="03a2b8f0-11c5-45c9-9107-ff8345de00c2" containerName="main"
Sep 18 20:15:00.200818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200625    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="620af298-7d5b-4723-9365-12e31947e124" containerName="maas-api"
Sep 18 20:15:00.200818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200633    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="620af298-7d5b-4723-9365-12e31947e124" containerName="maas-api"
Sep 18 20:15:00.200818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200657    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e21e30ad-89e2-474d-8614-6b330628eecb" containerName="payload-processing"
Sep 18 20:15:00.200818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200666    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e21e30ad-89e2-474d-8614-6b330628eecb" containerName="payload-processing"
Sep 18 20:15:00.200818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200678    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28eb230e-55ef-44e2-9676-7eb850504065" containerName="maas-api"
Sep 18 20:15:00.200818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200700    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="28eb230e-55ef-44e2-9676-7eb850504065" containerName="maas-api"
Sep 18 20:15:00.200818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200807    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="28eb230e-55ef-44e2-9676-7eb850504065" containerName="maas-api"
Sep 18 20:15:00.200818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200821    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="620af298-7d5b-4723-9365-12e31947e124" containerName="maas-api"
Sep 18 20:15:00.201222 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200832    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="03a2b8f0-11c5-45c9-9107-ff8345de00c2" containerName="main"
Sep 18 20:15:00.201222 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.200843    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e21e30ad-89e2-474d-8614-6b330628eecb" containerName="payload-processing"
Sep 18 20:15:00.202749 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.202719    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg6ss\" (UniqueName: \"kubernetes.io/projected/e21e30ad-89e2-474d-8614-6b330628eecb-kube-api-access-tg6ss\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:15:00.202749 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.202748    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q6z5\" (UniqueName: \"kubernetes.io/projected/620af298-7d5b-4723-9365-12e31947e124-kube-api-access-7q6z5\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:15:00.202961 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.202759    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-maas-api-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:15:00.202961 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.202767    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/620af298-7d5b-4723-9365-12e31947e124-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:15:00.202961 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.202775    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/620af298-7d5b-4723-9365-12e31947e124-openshift-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:15:00.206959 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.206935    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5"
Sep 18 20:15:00.209292 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.209263    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj"]
Sep 18 20:15:00.211556 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.211529    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bhssz\""
Sep 18 20:15:00.214079 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.214042    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5"]
Sep 18 20:15:00.222303 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.222277    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5"
Sep 18 20:15:00.235404 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.235373    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"]
Sep 18 20:15:00.240720 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.240669    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-4b633d-5b696bd5f7-nvjnq"]
Sep 18 20:15:00.519777 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.519743    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5"]
Sep 18 20:15:00.521238 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:15:00.521209    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb2641e_4cb0_4b8a_bca3_61bee7ea2166.slice/crio-52ee1dcec71a1c410a48d60323336af6e482307df2c44272820ae70d30f3e2cb WatchSource:0}: Error finding container 52ee1dcec71a1c410a48d60323336af6e482307df2c44272820ae70d30f3e2cb: Status 404 returned error can't find the container with id 52ee1dcec71a1c410a48d60323336af6e482307df2c44272820ae70d30f3e2cb
Sep 18 20:15:00.523084 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:00.523067    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:15:01.153132 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:01.153095    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5" event={"ID":"7bb2641e-4cb0-4b8a-bca3-61bee7ea2166","Type":"ContainerStarted","Data":"52ee1dcec71a1c410a48d60323336af6e482307df2c44272820ae70d30f3e2cb"}
Sep 18 20:15:01.702638 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:01.702604    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="620af298-7d5b-4723-9365-12e31947e124" path="/var/lib/kubelet/pods/620af298-7d5b-4723-9365-12e31947e124/volumes"
Sep 18 20:15:01.703114 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:01.703097    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e21e30ad-89e2-474d-8614-6b330628eecb" path="/var/lib/kubelet/pods/e21e30ad-89e2-474d-8614-6b330628eecb/volumes"
Sep 18 20:15:02.904047 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:02.903943    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-4b633d-5879d657b6-hq8gj" podUID="620af298-7d5b-4723-9365-12e31947e124" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.43:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 18 20:15:03.166105 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:03.166011    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5" event={"ID":"7bb2641e-4cb0-4b8a-bca3-61bee7ea2166","Type":"ContainerStarted","Data":"1e889c2f16fea475a42644660665efc70534a71a4ef26dbf00bd3e3d7a7a569a"}
Sep 18 20:15:03.184047 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:03.183989    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5" podStartSLOduration=1.079148082 podStartE2EDuration="3.183970376s" totalImagesPullingTime="2.104822294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:15:03.183059817 +0000 UTC m=+1083.743643275" watchObservedRunningTime="2026-09-18 20:15:03.183970376 +0000 UTC m=+1083.744553833"
Sep 18 20:15:04.173122 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:04.173087    2567 generic.go:356] "Generic (PLEG): container finished" podID="7bb2641e-4cb0-4b8a-bca3-61bee7ea2166" containerID="1e889c2f16fea475a42644660665efc70534a71a4ef26dbf00bd3e3d7a7a569a" exitCode=0
Sep 18 20:15:04.173542 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:04.173178    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5" event={"ID":"7bb2641e-4cb0-4b8a-bca3-61bee7ea2166","Type":"ContainerDied","Data":"1e889c2f16fea475a42644660665efc70534a71a4ef26dbf00bd3e3d7a7a569a"}
Sep 18 20:15:05.322375 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:05.322344    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5"
Sep 18 20:15:06.184820 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:06.184777    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5" event={"ID":"7bb2641e-4cb0-4b8a-bca3-61bee7ea2166","Type":"ContainerDied","Data":"52ee1dcec71a1c410a48d60323336af6e482307df2c44272820ae70d30f3e2cb"}
Sep 18 20:15:06.184820 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:06.184801    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5"
Sep 18 20:15:06.185045 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:06.184833    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="52ee1dcec71a1c410a48d60323336af6e482307df2c44272820ae70d30f3e2cb"
Sep 18 20:15:29.964752 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:29.964712    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"]
Sep 18 20:15:29.965357 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:29.965232    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bb2641e-4cb0-4b8a-bca3-61bee7ea2166" containerName="cleanup"
Sep 18 20:15:29.965357 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:29.965245    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bb2641e-4cb0-4b8a-bca3-61bee7ea2166" containerName="cleanup"
Sep 18 20:15:29.965357 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:29.965320    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bb2641e-4cb0-4b8a-bca3-61bee7ea2166" containerName="cleanup"
Sep 18 20:15:29.968841 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:29.968820    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:29.973278 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:29.973257    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-46f024\""
Sep 18 20:15:29.973493 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:29.973479    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-46f024-dockercfg-v5xk2\""
Sep 18 20:15:29.984243 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:29.984214    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"]
Sep 18 20:15:30.011837 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:30.011798    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556047a2-b7d8-4b85-a974-83d5d3f04bc6-plugins-config-volume\") pod \"payload-processing-e2e-explicit-46f024-55998f9b57-shlcb\" (UID: \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\") " pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:30.012021 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:30.011851    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9tjn\" (UniqueName: \"kubernetes.io/projected/556047a2-b7d8-4b85-a974-83d5d3f04bc6-kube-api-access-n9tjn\") pod \"payload-processing-e2e-explicit-46f024-55998f9b57-shlcb\" (UID: \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\") " pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:30.112389 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:30.112348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556047a2-b7d8-4b85-a974-83d5d3f04bc6-plugins-config-volume\") pod \"payload-processing-e2e-explicit-46f024-55998f9b57-shlcb\" (UID: \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\") " pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:30.112389 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:30.112397    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9tjn\" (UniqueName: \"kubernetes.io/projected/556047a2-b7d8-4b85-a974-83d5d3f04bc6-kube-api-access-n9tjn\") pod \"payload-processing-e2e-explicit-46f024-55998f9b57-shlcb\" (UID: \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\") " pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:30.113085 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:30.113064    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556047a2-b7d8-4b85-a974-83d5d3f04bc6-plugins-config-volume\") pod \"payload-processing-e2e-explicit-46f024-55998f9b57-shlcb\" (UID: \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\") " pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:30.122413 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:30.122375    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9tjn\" (UniqueName: \"kubernetes.io/projected/556047a2-b7d8-4b85-a974-83d5d3f04bc6-kube-api-access-n9tjn\") pod \"payload-processing-e2e-explicit-46f024-55998f9b57-shlcb\" (UID: \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\") " pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:30.280004 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:30.279908    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:30.564809 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:30.564778    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"]
Sep 18 20:15:30.566039 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:15:30.566011    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod556047a2_b7d8_4b85_a974_83d5d3f04bc6.slice/crio-a1b91b082d8d8be346e5f6802ea37de9b02ffb3928bd65bd04aed1c65dc284c3 WatchSource:0}: Error finding container a1b91b082d8d8be346e5f6802ea37de9b02ffb3928bd65bd04aed1c65dc284c3: Status 404 returned error can't find the container with id a1b91b082d8d8be346e5f6802ea37de9b02ffb3928bd65bd04aed1c65dc284c3
Sep 18 20:15:31.323221 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:31.323182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb" event={"ID":"556047a2-b7d8-4b85-a974-83d5d3f04bc6","Type":"ContainerStarted","Data":"5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a"}
Sep 18 20:15:31.323698 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:31.323246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb" event={"ID":"556047a2-b7d8-4b85-a974-83d5d3f04bc6","Type":"ContainerStarted","Data":"a1b91b082d8d8be346e5f6802ea37de9b02ffb3928bd65bd04aed1c65dc284c3"}
Sep 18 20:15:31.323698 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:31.323412    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:42.331857 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:42.331808    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:15:42.353049 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:15:42.352994    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb" podStartSLOduration=13.352976845 podStartE2EDuration="13.352976845s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:15:31.340701939 +0000 UTC m=+1111.901285396" watchObservedRunningTime="2026-09-18 20:15:42.352976845 +0000 UTC m=+1122.913560302"
Sep 18 20:16:25.021204 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.021167    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"]
Sep 18 20:16:25.021816 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.021437    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb" podUID="556047a2-b7d8-4b85-a974-83d5d3f04bc6" containerName="payload-processing" containerID="cri-o://5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a" gracePeriod=30
Sep 18 20:16:25.299119 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.299088    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:16:25.399613 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.399571    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9tjn\" (UniqueName: \"kubernetes.io/projected/556047a2-b7d8-4b85-a974-83d5d3f04bc6-kube-api-access-n9tjn\") pod \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\" (UID: \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\") "
Sep 18 20:16:25.399824 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.399729    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556047a2-b7d8-4b85-a974-83d5d3f04bc6-plugins-config-volume\") pod \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\" (UID: \"556047a2-b7d8-4b85-a974-83d5d3f04bc6\") "
Sep 18 20:16:25.400164 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.400139    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/556047a2-b7d8-4b85-a974-83d5d3f04bc6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "556047a2-b7d8-4b85-a974-83d5d3f04bc6" (UID: "556047a2-b7d8-4b85-a974-83d5d3f04bc6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:16:25.401935 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.401913    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/556047a2-b7d8-4b85-a974-83d5d3f04bc6-kube-api-access-n9tjn" (OuterVolumeSpecName: "kube-api-access-n9tjn") pod "556047a2-b7d8-4b85-a974-83d5d3f04bc6" (UID: "556047a2-b7d8-4b85-a974-83d5d3f04bc6"). InnerVolumeSpecName "kube-api-access-n9tjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:16:25.501394 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.501356    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/556047a2-b7d8-4b85-a974-83d5d3f04bc6-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:16:25.501394 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.501395    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9tjn\" (UniqueName: \"kubernetes.io/projected/556047a2-b7d8-4b85-a974-83d5d3f04bc6-kube-api-access-n9tjn\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:16:25.630585 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.630552    2567 generic.go:356] "Generic (PLEG): container finished" podID="556047a2-b7d8-4b85-a974-83d5d3f04bc6" containerID="5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a" exitCode=0
Sep 18 20:16:25.630840 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.630612    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"
Sep 18 20:16:25.630840 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.630633    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb" event={"ID":"556047a2-b7d8-4b85-a974-83d5d3f04bc6","Type":"ContainerDied","Data":"5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a"}
Sep 18 20:16:25.630840 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.630747    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb" event={"ID":"556047a2-b7d8-4b85-a974-83d5d3f04bc6","Type":"ContainerDied","Data":"a1b91b082d8d8be346e5f6802ea37de9b02ffb3928bd65bd04aed1c65dc284c3"}
Sep 18 20:16:25.630840 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.630779    2567 scope.go:117] "RemoveContainer" containerID="5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a"
Sep 18 20:16:25.640831 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.640803    2567 scope.go:117] "RemoveContainer" containerID="5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a"
Sep 18 20:16:25.641173 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:16:25.641150    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a\": container with ID starting with 5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a not found: ID does not exist" containerID="5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a"
Sep 18 20:16:25.641232 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.641183    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a"} err="failed to get container status \"5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a\": rpc error: code = NotFound desc = could not find container \"5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a\": container with ID starting with 5e28f13a453ff86ad5dab05fa6e1095f6568b229bd4f95719bd1488fde4db47a not found: ID does not exist"
Sep 18 20:16:25.688922 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.688889    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"]
Sep 18 20:16:25.700863 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:25.700831    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-46f024-55998f9b57-shlcb"]
Sep 18 20:16:27.699225 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:27.699190    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="556047a2-b7d8-4b85-a974-83d5d3f04bc6" path="/var/lib/kubelet/pods/556047a2-b7d8-4b85-a974-83d5d3f04bc6/volumes"
Sep 18 20:16:47.315725 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.315674    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"]
Sep 18 20:16:47.316182 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.316165    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="556047a2-b7d8-4b85-a974-83d5d3f04bc6" containerName="payload-processing"
Sep 18 20:16:47.316182 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.316184    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="556047a2-b7d8-4b85-a974-83d5d3f04bc6" containerName="payload-processing"
Sep 18 20:16:47.316282 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.316273    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="556047a2-b7d8-4b85-a974-83d5d3f04bc6" containerName="payload-processing"
Sep 18 20:16:47.320826 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.320808    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:16:47.326968 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.326943    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-aba3750c\""
Sep 18 20:16:47.343644 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.343608    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"]
Sep 18 20:16:47.411126 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.411084    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6327cb35-920c-4950-9283-b52f714a1702-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt\" (UID: \"6327cb35-920c-4950-9283-b52f714a1702\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:16:47.411126 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.411126    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw8qv\" (UniqueName: \"kubernetes.io/projected/6327cb35-920c-4950-9283-b52f714a1702-kube-api-access-lw8qv\") pod \"payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt\" (UID: \"6327cb35-920c-4950-9283-b52f714a1702\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:16:47.512325 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.512290    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6327cb35-920c-4950-9283-b52f714a1702-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt\" (UID: \"6327cb35-920c-4950-9283-b52f714a1702\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:16:47.512520 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.512335    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw8qv\" (UniqueName: \"kubernetes.io/projected/6327cb35-920c-4950-9283-b52f714a1702-kube-api-access-lw8qv\") pod \"payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt\" (UID: \"6327cb35-920c-4950-9283-b52f714a1702\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:16:47.513037 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.513013    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6327cb35-920c-4950-9283-b52f714a1702-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt\" (UID: \"6327cb35-920c-4950-9283-b52f714a1702\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:16:47.538223 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.538190    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw8qv\" (UniqueName: \"kubernetes.io/projected/6327cb35-920c-4950-9283-b52f714a1702-kube-api-access-lw8qv\") pod \"payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt\" (UID: \"6327cb35-920c-4950-9283-b52f714a1702\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:16:47.630983 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.630947    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:16:47.921074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:47.921018    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"]
Sep 18 20:16:47.923603 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:16:47.923571    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6327cb35_920c_4950_9283_b52f714a1702.slice/crio-d78b739a5ab96bd62f98a3a0a9cf6dab94263a27f68de3f36f7cb593a5773677 WatchSource:0}: Error finding container d78b739a5ab96bd62f98a3a0a9cf6dab94263a27f68de3f36f7cb593a5773677: Status 404 returned error can't find the container with id d78b739a5ab96bd62f98a3a0a9cf6dab94263a27f68de3f36f7cb593a5773677
Sep 18 20:16:48.770738 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:48.770702    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt" event={"ID":"6327cb35-920c-4950-9283-b52f714a1702","Type":"ContainerStarted","Data":"262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598"}
Sep 18 20:16:48.771214 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:48.770756    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt" event={"ID":"6327cb35-920c-4950-9283-b52f714a1702","Type":"ContainerStarted","Data":"d78b739a5ab96bd62f98a3a0a9cf6dab94263a27f68de3f36f7cb593a5773677"}
Sep 18 20:16:48.771214 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:48.770778    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:16:48.795299 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:48.795242    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt" podStartSLOduration=1.7952275800000002 podStartE2EDuration="1.79522758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:16:48.79437337 +0000 UTC m=+1189.354956827" watchObservedRunningTime="2026-09-18 20:16:48.79522758 +0000 UTC m=+1189.355811036"
Sep 18 20:16:59.779233 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:16:59.779192    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:17:01.720253 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:01.720224    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:17:01.724373 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:01.724347    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:17:04.319202 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.319166    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"]
Sep 18 20:17:04.324139 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.324121    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.327710 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.327659    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-a733e2\"/\"openshift-service-ca.crt\""
Sep 18 20:17:04.328362 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.328347    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-a733e2\"/\"kube-root-ca.crt\""
Sep 18 20:17:04.328518 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.328501    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-a733e2\"/\"default-dockercfg-vsgm5\""
Sep 18 20:17:04.329027 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.329011    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-a733e2\"/\"noten-model-a733e2-kserve-self-signed-certs\""
Sep 18 20:17:04.337427 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.337404    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"]
Sep 18 20:17:04.492982 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.492946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-dshm\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.493169 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.493042    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-model-cache\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.493169 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.493072    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h8z2\" (UniqueName: \"kubernetes.io/projected/8105b45e-14cb-4149-818a-0755de55f39d-kube-api-access-5h8z2\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.493169 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.493104    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-tmp-dir\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.493306 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.493176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-home\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.493306 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.493195    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8105b45e-14cb-4149-818a-0755de55f39d-tls-certs\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.594333 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.594224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-home\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.594333 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.594270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8105b45e-14cb-4149-818a-0755de55f39d-tls-certs\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.594589 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.594395    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-dshm\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.594589 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.594487    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-model-cache\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.594589 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.594512    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h8z2\" (UniqueName: \"kubernetes.io/projected/8105b45e-14cb-4149-818a-0755de55f39d-kube-api-access-5h8z2\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.594589 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.594552    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-tmp-dir\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.594836 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.594740    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-home\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.594904 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.594881    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-model-cache\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.594949 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.594914    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-tmp-dir\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.596638 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.596608    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-dshm\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.596924 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.596906    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8105b45e-14cb-4149-818a-0755de55f39d-tls-certs\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.604212 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.604188    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h8z2\" (UniqueName: \"kubernetes.io/projected/8105b45e-14cb-4149-818a-0755de55f39d-kube-api-access-5h8z2\") pod \"noten-model-a733e2-kserve-c7b66dbf9-flbz5\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") " pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.636213 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.636167    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:04.923993 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:04.923961    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"]
Sep 18 20:17:04.925306 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:17:04.925282    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8105b45e_14cb_4149_818a_0755de55f39d.slice/crio-5f87be0af23608a997589d478bf24ce9e799d923e435c6d1e55e9255f2dc09b0 WatchSource:0}: Error finding container 5f87be0af23608a997589d478bf24ce9e799d923e435c6d1e55e9255f2dc09b0: Status 404 returned error can't find the container with id 5f87be0af23608a997589d478bf24ce9e799d923e435c6d1e55e9255f2dc09b0
Sep 18 20:17:05.866343 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:05.866306    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5" event={"ID":"8105b45e-14cb-4149-818a-0755de55f39d","Type":"ContainerStarted","Data":"dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158"}
Sep 18 20:17:05.866771 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:05.866376    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5" event={"ID":"8105b45e-14cb-4149-818a-0755de55f39d","Type":"ContainerStarted","Data":"5f87be0af23608a997589d478bf24ce9e799d923e435c6d1e55e9255f2dc09b0"}
Sep 18 20:17:05.887658 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:05.887609    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5" podStartSLOduration=1.887592513 podStartE2EDuration="1.887592513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:05.88611127 +0000 UTC m=+1206.446694727" watchObservedRunningTime="2026-09-18 20:17:05.887592513 +0000 UTC m=+1206.448175967"
Sep 18 20:17:14.637076 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:14.637040    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:14.637562 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:14.637097    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:14.650157 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:14.650123    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:14.937992 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:14.937900    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:20.883919 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:20.883877    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"]
Sep 18 20:17:22.189482 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:22.189442    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"]
Sep 18 20:17:33.231753 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:33.231714    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"]
Sep 18 20:17:33.232389 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:33.231966    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt" podUID="6327cb35-920c-4950-9283-b52f714a1702" containerName="payload-pre-processing" containerID="cri-o://262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598" gracePeriod=30
Sep 18 20:17:33.499965 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:33.499884    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:17:33.595108 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:33.595070    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lw8qv\" (UniqueName: \"kubernetes.io/projected/6327cb35-920c-4950-9283-b52f714a1702-kube-api-access-lw8qv\") pod \"6327cb35-920c-4950-9283-b52f714a1702\" (UID: \"6327cb35-920c-4950-9283-b52f714a1702\") "
Sep 18 20:17:33.595292 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:33.595197    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6327cb35-920c-4950-9283-b52f714a1702-plugins-config-volume\") pod \"6327cb35-920c-4950-9283-b52f714a1702\" (UID: \"6327cb35-920c-4950-9283-b52f714a1702\") "
Sep 18 20:17:33.595590 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:33.595560    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6327cb35-920c-4950-9283-b52f714a1702-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6327cb35-920c-4950-9283-b52f714a1702" (UID: "6327cb35-920c-4950-9283-b52f714a1702"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:17:33.597400 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:33.597370    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6327cb35-920c-4950-9283-b52f714a1702-kube-api-access-lw8qv" (OuterVolumeSpecName: "kube-api-access-lw8qv") pod "6327cb35-920c-4950-9283-b52f714a1702" (UID: "6327cb35-920c-4950-9283-b52f714a1702"). InnerVolumeSpecName "kube-api-access-lw8qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:17:33.696211 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:33.696178    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6327cb35-920c-4950-9283-b52f714a1702-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:33.696211 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:33.696208    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw8qv\" (UniqueName: \"kubernetes.io/projected/6327cb35-920c-4950-9283-b52f714a1702-kube-api-access-lw8qv\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:34.033637 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:34.033598    2567 generic.go:356] "Generic (PLEG): container finished" podID="6327cb35-920c-4950-9283-b52f714a1702" containerID="262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598" exitCode=0
Sep 18 20:17:34.033848 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:34.033641    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"
Sep 18 20:17:34.033848 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:34.033710    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt" event={"ID":"6327cb35-920c-4950-9283-b52f714a1702","Type":"ContainerDied","Data":"262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598"}
Sep 18 20:17:34.033848 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:34.033767    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt" event={"ID":"6327cb35-920c-4950-9283-b52f714a1702","Type":"ContainerDied","Data":"d78b739a5ab96bd62f98a3a0a9cf6dab94263a27f68de3f36f7cb593a5773677"}
Sep 18 20:17:34.033848 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:34.033789    2567 scope.go:117] "RemoveContainer" containerID="262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598"
Sep 18 20:17:34.043558 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:34.043537    2567 scope.go:117] "RemoveContainer" containerID="262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598"
Sep 18 20:17:34.043897 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:17:34.043877    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598\": container with ID starting with 262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598 not found: ID does not exist" containerID="262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598"
Sep 18 20:17:34.043988 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:34.043905    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598"} err="failed to get container status \"262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598\": rpc error: code = NotFound desc = could not find container \"262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598\": container with ID starting with 262fa5eae0394cb40e6dc5ce8f9cc968bc0807ed607b8ebc9a8f8fbc5cf80598 not found: ID does not exist"
Sep 18 20:17:34.057584 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:34.057552    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"]
Sep 18 20:17:34.062314 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:34.062284    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-aba3750c-55f999b9fd-l47xt"]
Sep 18 20:17:35.699892 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:35.699858    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6327cb35-920c-4950-9283-b52f714a1702" path="/var/lib/kubelet/pods/6327cb35-920c-4950-9283-b52f714a1702/volumes"
Sep 18 20:17:35.903501 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:35.903459    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn" podUID="46b59d31-a96b-4993-ba45-80b8d7b89fae" containerName="main" containerID="cri-o://5d6e0433b5bce6255b6a0796da2d913a344e6dbc5f4d27fbc61d3d070d6526d3" gracePeriod=45
Sep 18 20:17:36.051200 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.051171    2567 generic.go:356] "Generic (PLEG): container finished" podID="46b59d31-a96b-4993-ba45-80b8d7b89fae" containerID="5d6e0433b5bce6255b6a0796da2d913a344e6dbc5f4d27fbc61d3d070d6526d3" exitCode=0
Sep 18 20:17:36.051365 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.051239    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn" event={"ID":"46b59d31-a96b-4993-ba45-80b8d7b89fae","Type":"ContainerDied","Data":"5d6e0433b5bce6255b6a0796da2d913a344e6dbc5f4d27fbc61d3d070d6526d3"}
Sep 18 20:17:36.167823 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.167797    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:17:36.323375 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323285    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-tmp-dir\") pod \"46b59d31-a96b-4993-ba45-80b8d7b89fae\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") "
Sep 18 20:17:36.323375 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323335    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-home\") pod \"46b59d31-a96b-4993-ba45-80b8d7b89fae\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") "
Sep 18 20:17:36.323607 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323399    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cg46\" (UniqueName: \"kubernetes.io/projected/46b59d31-a96b-4993-ba45-80b8d7b89fae-kube-api-access-6cg46\") pod \"46b59d31-a96b-4993-ba45-80b8d7b89fae\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") "
Sep 18 20:17:36.323607 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323420    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-dshm\") pod \"46b59d31-a96b-4993-ba45-80b8d7b89fae\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") "
Sep 18 20:17:36.323607 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323534    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46b59d31-a96b-4993-ba45-80b8d7b89fae-tls-certs\") pod \"46b59d31-a96b-4993-ba45-80b8d7b89fae\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") "
Sep 18 20:17:36.323607 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323600    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-model-cache\") pod \"46b59d31-a96b-4993-ba45-80b8d7b89fae\" (UID: \"46b59d31-a96b-4993-ba45-80b8d7b89fae\") "
Sep 18 20:17:36.323861 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323613    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "46b59d31-a96b-4993-ba45-80b8d7b89fae" (UID: "46b59d31-a96b-4993-ba45-80b8d7b89fae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:36.323861 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323681    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-home" (OuterVolumeSpecName: "home") pod "46b59d31-a96b-4993-ba45-80b8d7b89fae" (UID: "46b59d31-a96b-4993-ba45-80b8d7b89fae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:36.323960 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323894    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-model-cache" (OuterVolumeSpecName: "model-cache") pod "46b59d31-a96b-4993-ba45-80b8d7b89fae" (UID: "46b59d31-a96b-4993-ba45-80b8d7b89fae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:36.324001 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323969    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-model-cache\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:36.324001 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.323986    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-tmp-dir\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:36.324072 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.324006    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-home\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:36.325839 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.325814    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46b59d31-a96b-4993-ba45-80b8d7b89fae-kube-api-access-6cg46" (OuterVolumeSpecName: "kube-api-access-6cg46") pod "46b59d31-a96b-4993-ba45-80b8d7b89fae" (UID: "46b59d31-a96b-4993-ba45-80b8d7b89fae"). InnerVolumeSpecName "kube-api-access-6cg46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:17:36.325927 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.325818    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-dshm" (OuterVolumeSpecName: "dshm") pod "46b59d31-a96b-4993-ba45-80b8d7b89fae" (UID: "46b59d31-a96b-4993-ba45-80b8d7b89fae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:36.325927 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.325863    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b59d31-a96b-4993-ba45-80b8d7b89fae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "46b59d31-a96b-4993-ba45-80b8d7b89fae" (UID: "46b59d31-a96b-4993-ba45-80b8d7b89fae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:17:36.424836 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.424800    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46b59d31-a96b-4993-ba45-80b8d7b89fae-tls-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:36.424836 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.424833    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cg46\" (UniqueName: \"kubernetes.io/projected/46b59d31-a96b-4993-ba45-80b8d7b89fae-kube-api-access-6cg46\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:36.424836 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:36.424843    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46b59d31-a96b-4993-ba45-80b8d7b89fae-dshm\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:37.058271 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.058238    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"
Sep 18 20:17:37.058782 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.058265    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn" event={"ID":"46b59d31-a96b-4993-ba45-80b8d7b89fae","Type":"ContainerDied","Data":"56fff35fbef5640df655fb78840352afb6087e0d7e7e44322c9982622967afed"}
Sep 18 20:17:37.058782 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.058317    2567 scope.go:117] "RemoveContainer" containerID="5d6e0433b5bce6255b6a0796da2d913a344e6dbc5f4d27fbc61d3d070d6526d3"
Sep 18 20:17:37.168819 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.168783    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"]
Sep 18 20:17:37.176592 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.176565    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-3a8052/test-model-3a8052-kserve-867c6b6cdc-4vjrn"]
Sep 18 20:17:37.210487 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.210433    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5" podUID="8105b45e-14cb-4149-818a-0755de55f39d" containerName="main" containerID="cri-o://dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158" gracePeriod=45
Sep 18 20:17:37.460843 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.460817    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:37.637028 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.636997    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-tmp-dir\") pod \"8105b45e-14cb-4149-818a-0755de55f39d\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") "
Sep 18 20:17:37.637210 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637044    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h8z2\" (UniqueName: \"kubernetes.io/projected/8105b45e-14cb-4149-818a-0755de55f39d-kube-api-access-5h8z2\") pod \"8105b45e-14cb-4149-818a-0755de55f39d\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") "
Sep 18 20:17:37.637210 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637066    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-dshm\") pod \"8105b45e-14cb-4149-818a-0755de55f39d\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") "
Sep 18 20:17:37.637210 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637103    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-model-cache\") pod \"8105b45e-14cb-4149-818a-0755de55f39d\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") "
Sep 18 20:17:37.637210 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637164    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8105b45e-14cb-4149-818a-0755de55f39d-tls-certs\") pod \"8105b45e-14cb-4149-818a-0755de55f39d\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") "
Sep 18 20:17:37.637210 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637199    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-home\") pod \"8105b45e-14cb-4149-818a-0755de55f39d\" (UID: \"8105b45e-14cb-4149-818a-0755de55f39d\") "
Sep 18 20:17:37.637478 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637352    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8105b45e-14cb-4149-818a-0755de55f39d" (UID: "8105b45e-14cb-4149-818a-0755de55f39d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:37.637478 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637391    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-model-cache" (OuterVolumeSpecName: "model-cache") pod "8105b45e-14cb-4149-818a-0755de55f39d" (UID: "8105b45e-14cb-4149-818a-0755de55f39d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:37.637583 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637515    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-tmp-dir\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:37.637583 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637536    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-model-cache\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:37.637583 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.637544    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-home" (OuterVolumeSpecName: "home") pod "8105b45e-14cb-4149-818a-0755de55f39d" (UID: "8105b45e-14cb-4149-818a-0755de55f39d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:37.639380 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.639346    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-dshm" (OuterVolumeSpecName: "dshm") pod "8105b45e-14cb-4149-818a-0755de55f39d" (UID: "8105b45e-14cb-4149-818a-0755de55f39d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:17:37.639380 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.639357    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8105b45e-14cb-4149-818a-0755de55f39d-kube-api-access-5h8z2" (OuterVolumeSpecName: "kube-api-access-5h8z2") pod "8105b45e-14cb-4149-818a-0755de55f39d" (UID: "8105b45e-14cb-4149-818a-0755de55f39d"). InnerVolumeSpecName "kube-api-access-5h8z2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:17:37.639380 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.639374    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8105b45e-14cb-4149-818a-0755de55f39d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8105b45e-14cb-4149-818a-0755de55f39d" (UID: "8105b45e-14cb-4149-818a-0755de55f39d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:17:37.700342 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.700307    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46b59d31-a96b-4993-ba45-80b8d7b89fae" path="/var/lib/kubelet/pods/46b59d31-a96b-4993-ba45-80b8d7b89fae/volumes"
Sep 18 20:17:37.738093 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.738056    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-dshm\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:37.738093 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.738085    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8105b45e-14cb-4149-818a-0755de55f39d-tls-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:37.738093 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.738096    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8105b45e-14cb-4149-818a-0755de55f39d-home\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:37.738324 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:37.738107    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h8z2\" (UniqueName: \"kubernetes.io/projected/8105b45e-14cb-4149-818a-0755de55f39d-kube-api-access-5h8z2\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:38.066785 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:38.066669    2567 generic.go:356] "Generic (PLEG): container finished" podID="8105b45e-14cb-4149-818a-0755de55f39d" containerID="dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158" exitCode=0
Sep 18 20:17:38.066785 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:38.066728    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"
Sep 18 20:17:38.067237 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:38.066724    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5" event={"ID":"8105b45e-14cb-4149-818a-0755de55f39d","Type":"ContainerDied","Data":"dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158"}
Sep 18 20:17:38.067237 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:38.066880    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5" event={"ID":"8105b45e-14cb-4149-818a-0755de55f39d","Type":"ContainerDied","Data":"5f87be0af23608a997589d478bf24ce9e799d923e435c6d1e55e9255f2dc09b0"}
Sep 18 20:17:38.067237 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:38.066900    2567 scope.go:117] "RemoveContainer" containerID="dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158"
Sep 18 20:17:38.078356 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:38.078326    2567 scope.go:117] "RemoveContainer" containerID="dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158"
Sep 18 20:17:38.078768 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:17:38.078745    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158\": container with ID starting with dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158 not found: ID does not exist" containerID="dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158"
Sep 18 20:17:38.078885 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:38.078777    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158"} err="failed to get container status \"dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158\": rpc error: code = NotFound desc = could not find container \"dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158\": container with ID starting with dc8ca3e2d5f65e718e64c8a78bb522059deb6c94736134709fda4edf1451b158 not found: ID does not exist"
Sep 18 20:17:38.116751 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:38.116705    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"]
Sep 18 20:17:38.124890 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:38.124857    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-a733e2/noten-model-a733e2-kserve-c7b66dbf9-flbz5"]
Sep 18 20:17:39.701587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:39.701542    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8105b45e-14cb-4149-818a-0755de55f39d" path="/var/lib/kubelet/pods/8105b45e-14cb-4149-818a-0755de55f39d/volumes"
Sep 18 20:17:46.466189 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:46.466152    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"]
Sep 18 20:17:46.468651 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:46.466597    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd" podUID="ebe658fa-e679-409e-b5a3-00fca0138cb6" containerName="payload-pre-processing" containerID="cri-o://f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7" gracePeriod=30
Sep 18 20:17:46.724941 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:46.724875    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:17:46.834109 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:46.834074    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe658fa-e679-409e-b5a3-00fca0138cb6-plugins-config-volume\") pod \"ebe658fa-e679-409e-b5a3-00fca0138cb6\" (UID: \"ebe658fa-e679-409e-b5a3-00fca0138cb6\") "
Sep 18 20:17:46.834318 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:46.834148    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jdsv\" (UniqueName: \"kubernetes.io/projected/ebe658fa-e679-409e-b5a3-00fca0138cb6-kube-api-access-6jdsv\") pod \"ebe658fa-e679-409e-b5a3-00fca0138cb6\" (UID: \"ebe658fa-e679-409e-b5a3-00fca0138cb6\") "
Sep 18 20:17:46.834486 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:46.834460    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebe658fa-e679-409e-b5a3-00fca0138cb6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ebe658fa-e679-409e-b5a3-00fca0138cb6" (UID: "ebe658fa-e679-409e-b5a3-00fca0138cb6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:17:46.836284 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:46.836258    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebe658fa-e679-409e-b5a3-00fca0138cb6-kube-api-access-6jdsv" (OuterVolumeSpecName: "kube-api-access-6jdsv") pod "ebe658fa-e679-409e-b5a3-00fca0138cb6" (UID: "ebe658fa-e679-409e-b5a3-00fca0138cb6"). InnerVolumeSpecName "kube-api-access-6jdsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:17:46.935557 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:46.935504    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebe658fa-e679-409e-b5a3-00fca0138cb6-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:46.935557 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:46.935557    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jdsv\" (UniqueName: \"kubernetes.io/projected/ebe658fa-e679-409e-b5a3-00fca0138cb6-kube-api-access-6jdsv\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:17:47.121673 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.121635    2567 generic.go:356] "Generic (PLEG): container finished" podID="ebe658fa-e679-409e-b5a3-00fca0138cb6" containerID="f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7" exitCode=0
Sep 18 20:17:47.121894 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.121694    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"
Sep 18 20:17:47.121894 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.121721    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd" event={"ID":"ebe658fa-e679-409e-b5a3-00fca0138cb6","Type":"ContainerDied","Data":"f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7"}
Sep 18 20:17:47.121894 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.121781    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd" event={"ID":"ebe658fa-e679-409e-b5a3-00fca0138cb6","Type":"ContainerDied","Data":"b9419706612cc0884b1b48827e65b772d1c6df106c8d889f6dc5ca22e5176d18"}
Sep 18 20:17:47.121894 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.121800    2567 scope.go:117] "RemoveContainer" containerID="f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7"
Sep 18 20:17:47.131155 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.131137    2567 scope.go:117] "RemoveContainer" containerID="f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7"
Sep 18 20:17:47.131433 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:17:47.131414    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7\": container with ID starting with f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7 not found: ID does not exist" containerID="f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7"
Sep 18 20:17:47.131488 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.131440    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7"} err="failed to get container status \"f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7\": rpc error: code = NotFound desc = could not find container \"f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7\": container with ID starting with f2feb9de2948fcf99fbb26956ec986729076e3f20159f528cbf941f68e0957c7 not found: ID does not exist"
Sep 18 20:17:47.234148 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.234113    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"]
Sep 18 20:17:47.236378 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.236348    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-3a8052-6f755cc7c8-ll6nd"]
Sep 18 20:17:47.699702 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:47.699665    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebe658fa-e679-409e-b5a3-00fca0138cb6" path="/var/lib/kubelet/pods/ebe658fa-e679-409e-b5a3-00fca0138cb6/volumes"
Sep 18 20:17:58.594007 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.593969    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"]
Sep 18 20:17:58.594587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594435    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46b59d31-a96b-4993-ba45-80b8d7b89fae" containerName="main"
Sep 18 20:17:58.594587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594450    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="46b59d31-a96b-4993-ba45-80b8d7b89fae" containerName="main"
Sep 18 20:17:58.594587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594473    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8105b45e-14cb-4149-818a-0755de55f39d" containerName="main"
Sep 18 20:17:58.594587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594478    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8105b45e-14cb-4149-818a-0755de55f39d" containerName="main"
Sep 18 20:17:58.594587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594486    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebe658fa-e679-409e-b5a3-00fca0138cb6" containerName="payload-pre-processing"
Sep 18 20:17:58.594587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594492    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebe658fa-e679-409e-b5a3-00fca0138cb6" containerName="payload-pre-processing"
Sep 18 20:17:58.594587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594500    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6327cb35-920c-4950-9283-b52f714a1702" containerName="payload-pre-processing"
Sep 18 20:17:58.594587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594509    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6327cb35-920c-4950-9283-b52f714a1702" containerName="payload-pre-processing"
Sep 18 20:17:58.595076 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594601    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebe658fa-e679-409e-b5a3-00fca0138cb6" containerName="payload-pre-processing"
Sep 18 20:17:58.595076 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594609    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="46b59d31-a96b-4993-ba45-80b8d7b89fae" containerName="main"
Sep 18 20:17:58.595076 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594617    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6327cb35-920c-4950-9283-b52f714a1702" containerName="payload-pre-processing"
Sep 18 20:17:58.595076 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.594626    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8105b45e-14cb-4149-818a-0755de55f39d" containerName="main"
Sep 18 20:17:58.597791 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.597772    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.603160 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.603141    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-5284dc\"/\"default-dockercfg-r2p8h\""
Sep 18 20:17:58.603843 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.603826    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-5284dc\"/\"openshift-service-ca.crt\""
Sep 18 20:17:58.604812 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.604788    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-5284dc\"/\"distinct-2-w0-5284dc-kserve-self-signed-certs\""
Sep 18 20:17:58.604917 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.604853    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-5284dc\"/\"kube-root-ca.crt\""
Sep 18 20:17:58.630800 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.630771    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"]
Sep 18 20:17:58.659892 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.659857    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-home\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.660066 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.659916    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-dshm\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.660066 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.660021    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-model-cache\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.660169 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.660077    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2kv2\" (UniqueName: \"kubernetes.io/projected/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-kube-api-access-f2kv2\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.660169 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.660100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tmp-dir\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.660169 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.660128    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tls-certs\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.761517 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.761483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-home\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.761739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.761539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-dshm\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.761739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.761600    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-model-cache\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.761739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.761641    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2kv2\" (UniqueName: \"kubernetes.io/projected/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-kube-api-access-f2kv2\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.761739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.761659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tmp-dir\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.761739 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.761713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tls-certs\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.762070 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.761994    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-home\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.762133 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.762060    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-model-cache\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.762191 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.762150    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tmp-dir\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.763925 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.763905    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-dshm\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.764056 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.764040    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tls-certs\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.778125 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.778093    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2kv2\" (UniqueName: \"kubernetes.io/projected/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-kube-api-access-f2kv2\") pod \"distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") " pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:58.907476 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:58.907436    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:17:59.214309 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:17:59.214284    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"]
Sep 18 20:17:59.216310 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:17:59.216275    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4087693a_d020_4fa8_a6f5_5fa2f56ee2a6.slice/crio-f8bef1db502428565290d499da057d243ff518ab351b72f80947ba91f2ff5274 WatchSource:0}: Error finding container f8bef1db502428565290d499da057d243ff518ab351b72f80947ba91f2ff5274: Status 404 returned error can't find the container with id f8bef1db502428565290d499da057d243ff518ab351b72f80947ba91f2ff5274
Sep 18 20:18:00.197153 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:00.197120    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq" event={"ID":"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6","Type":"ContainerStarted","Data":"417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a"}
Sep 18 20:18:00.197593 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:00.197165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq" event={"ID":"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6","Type":"ContainerStarted","Data":"f8bef1db502428565290d499da057d243ff518ab351b72f80947ba91f2ff5274"}
Sep 18 20:18:00.247295 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:00.247235    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq" podStartSLOduration=2.247216361 podStartE2EDuration="2.247216361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:18:00.245451074 +0000 UTC m=+1260.806034531" watchObservedRunningTime="2026-09-18 20:18:00.247216361 +0000 UTC m=+1260.807799874"
Sep 18 20:18:08.907649 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:08.907618    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:18:08.908055 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:08.907662    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:18:08.921368 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:08.921339    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:18:09.260552 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:09.260458    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:18:12.529288 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.529256    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"]
Sep 18 20:18:12.534407 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.534387    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:12.538212 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.538194    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-5319c6\""
Sep 18 20:18:12.554574 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.554545    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"]
Sep 18 20:18:12.704294 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.704253    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm5wk\" (UniqueName: \"kubernetes.io/projected/8c528839-a6fc-4085-b779-87826bb6588f-kube-api-access-pm5wk\") pod \"payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d\" (UID: \"8c528839-a6fc-4085-b779-87826bb6588f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:12.704481 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.704310    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c528839-a6fc-4085-b779-87826bb6588f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d\" (UID: \"8c528839-a6fc-4085-b779-87826bb6588f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:12.805802 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.805665    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm5wk\" (UniqueName: \"kubernetes.io/projected/8c528839-a6fc-4085-b779-87826bb6588f-kube-api-access-pm5wk\") pod \"payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d\" (UID: \"8c528839-a6fc-4085-b779-87826bb6588f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:12.805998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.805867    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c528839-a6fc-4085-b779-87826bb6588f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d\" (UID: \"8c528839-a6fc-4085-b779-87826bb6588f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:12.806613 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.806591    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c528839-a6fc-4085-b779-87826bb6588f-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d\" (UID: \"8c528839-a6fc-4085-b779-87826bb6588f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:12.817461 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.817430    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm5wk\" (UniqueName: \"kubernetes.io/projected/8c528839-a6fc-4085-b779-87826bb6588f-kube-api-access-pm5wk\") pod \"payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d\" (UID: \"8c528839-a6fc-4085-b779-87826bb6588f\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:12.845384 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.845354    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:12.877457 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.877418    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"]
Sep 18 20:18:12.877642 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.877612    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x" podUID="83f8e21b-264a-4a6f-a823-b6878475b2cd" containerName="maas-api" containerID="cri-o://d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674" gracePeriod=30
Sep 18 20:18:12.962828 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.962789    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5"]
Sep 18 20:18:12.974072 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:12.974041    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-c34fad-29829375-zp4g5"]
Sep 18 20:18:13.033313 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.033197    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"]
Sep 18 20:18:13.033493 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.033429    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s" podUID="9353cf66-a8b0-4281-86b1-90d2b12dc37a" containerName="payload-processing" containerID="cri-o://dcfcc425efd384fdc6e83ad7efb47b3265e7b6213dd1761e231152a036b3dec5" gracePeriod=30
Sep 18 20:18:13.147088 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.147063    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:18:13.159969 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.159945    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"]
Sep 18 20:18:13.163577 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:18:13.163534    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c528839_a6fc_4085_b779_87826bb6588f.slice/crio-8fbddfa0659920c4467f8dbfcc141be1eecde69cb6ae92452c217d7506e069e8 WatchSource:0}: Error finding container 8fbddfa0659920c4467f8dbfcc141be1eecde69cb6ae92452c217d7506e069e8: Status 404 returned error can't find the container with id 8fbddfa0659920c4467f8dbfcc141be1eecde69cb6ae92452c217d7506e069e8
Sep 18 20:18:13.275599 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.275563    2567 generic.go:356] "Generic (PLEG): container finished" podID="9353cf66-a8b0-4281-86b1-90d2b12dc37a" containerID="dcfcc425efd384fdc6e83ad7efb47b3265e7b6213dd1761e231152a036b3dec5" exitCode=0
Sep 18 20:18:13.275804 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.275594    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s" event={"ID":"9353cf66-a8b0-4281-86b1-90d2b12dc37a","Type":"ContainerDied","Data":"dcfcc425efd384fdc6e83ad7efb47b3265e7b6213dd1761e231152a036b3dec5"}
Sep 18 20:18:13.277222 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.277195    2567 generic.go:356] "Generic (PLEG): container finished" podID="83f8e21b-264a-4a6f-a823-b6878475b2cd" containerID="d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674" exitCode=0
Sep 18 20:18:13.277354 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.277257    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"
Sep 18 20:18:13.277354 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.277272    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x" event={"ID":"83f8e21b-264a-4a6f-a823-b6878475b2cd","Type":"ContainerDied","Data":"d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674"}
Sep 18 20:18:13.277354 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.277327    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x" event={"ID":"83f8e21b-264a-4a6f-a823-b6878475b2cd","Type":"ContainerDied","Data":"16b39f81b7fe0a81354130cf09a357e0ab062f15d806600093373393d8468317"}
Sep 18 20:18:13.277493 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.277358    2567 scope.go:117] "RemoveContainer" containerID="d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674"
Sep 18 20:18:13.279138 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.279110    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d" event={"ID":"8c528839-a6fc-4085-b779-87826bb6588f","Type":"ContainerStarted","Data":"08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9"}
Sep 18 20:18:13.279269 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.279165    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d" event={"ID":"8c528839-a6fc-4085-b779-87826bb6588f","Type":"ContainerStarted","Data":"8fbddfa0659920c4467f8dbfcc141be1eecde69cb6ae92452c217d7506e069e8"}
Sep 18 20:18:13.279269 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.279196    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:13.288062 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.288045    2567 scope.go:117] "RemoveContainer" containerID="d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674"
Sep 18 20:18:13.288369 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:18:13.288348    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674\": container with ID starting with d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674 not found: ID does not exist" containerID="d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674"
Sep 18 20:18:13.288443 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.288381    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674"} err="failed to get container status \"d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674\": rpc error: code = NotFound desc = could not find container \"d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674\": container with ID starting with d4883154fb449fdb5f10bed0affca2664c8712ca7292dc719d073e3584010674 not found: ID does not exist"
Sep 18 20:18:13.289355 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.289338    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:18:13.313950 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.313925    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-metrics-certs\") pod \"83f8e21b-264a-4a6f-a823-b6878475b2cd\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") "
Sep 18 20:18:13.314098 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.313996    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83f8e21b-264a-4a6f-a823-b6878475b2cd-openshift-service-ca\") pod \"83f8e21b-264a-4a6f-a823-b6878475b2cd\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") "
Sep 18 20:18:13.314098 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.314023    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-maas-api-tls\") pod \"83f8e21b-264a-4a6f-a823-b6878475b2cd\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") "
Sep 18 20:18:13.314098 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.314068    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq8d2\" (UniqueName: \"kubernetes.io/projected/83f8e21b-264a-4a6f-a823-b6878475b2cd-kube-api-access-jq8d2\") pod \"83f8e21b-264a-4a6f-a823-b6878475b2cd\" (UID: \"83f8e21b-264a-4a6f-a823-b6878475b2cd\") "
Sep 18 20:18:13.314399 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.314377    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83f8e21b-264a-4a6f-a823-b6878475b2cd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83f8e21b-264a-4a6f-a823-b6878475b2cd" (UID: "83f8e21b-264a-4a6f-a823-b6878475b2cd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:18:13.316291 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.316258    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83f8e21b-264a-4a6f-a823-b6878475b2cd-kube-api-access-jq8d2" (OuterVolumeSpecName: "kube-api-access-jq8d2") pod "83f8e21b-264a-4a6f-a823-b6878475b2cd" (UID: "83f8e21b-264a-4a6f-a823-b6878475b2cd"). InnerVolumeSpecName "kube-api-access-jq8d2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:18:13.316560 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.316538    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83f8e21b-264a-4a6f-a823-b6878475b2cd" (UID: "83f8e21b-264a-4a6f-a823-b6878475b2cd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:18:13.316630 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.316554    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "83f8e21b-264a-4a6f-a823-b6878475b2cd" (UID: "83f8e21b-264a-4a6f-a823-b6878475b2cd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:18:13.415243 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.415142    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phzl9\" (UniqueName: \"kubernetes.io/projected/9353cf66-a8b0-4281-86b1-90d2b12dc37a-kube-api-access-phzl9\") pod \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\" (UID: \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\") "
Sep 18 20:18:13.415243 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.415215    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9353cf66-a8b0-4281-86b1-90d2b12dc37a-plugins-config-volume\") pod \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\" (UID: \"9353cf66-a8b0-4281-86b1-90d2b12dc37a\") "
Sep 18 20:18:13.415633 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.415614    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.415769 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.415635    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83f8e21b-264a-4a6f-a823-b6878475b2cd-openshift-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.415769 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.415650    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83f8e21b-264a-4a6f-a823-b6878475b2cd-maas-api-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.415769 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.415664    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq8d2\" (UniqueName: \"kubernetes.io/projected/83f8e21b-264a-4a6f-a823-b6878475b2cd-kube-api-access-jq8d2\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.415941 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.415811    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9353cf66-a8b0-4281-86b1-90d2b12dc37a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9353cf66-a8b0-4281-86b1-90d2b12dc37a" (UID: "9353cf66-a8b0-4281-86b1-90d2b12dc37a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:18:13.417587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.417559    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9353cf66-a8b0-4281-86b1-90d2b12dc37a-kube-api-access-phzl9" (OuterVolumeSpecName: "kube-api-access-phzl9") pod "9353cf66-a8b0-4281-86b1-90d2b12dc37a" (UID: "9353cf66-a8b0-4281-86b1-90d2b12dc37a"). InnerVolumeSpecName "kube-api-access-phzl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:18:13.441169 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.441101    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d" podStartSLOduration=1.441076578 podStartE2EDuration="1.441076578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:18:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:18:13.366836809 +0000 UTC m=+1273.927420268" watchObservedRunningTime="2026-09-18 20:18:13.441076578 +0000 UTC m=+1274.001660039"
Sep 18 20:18:13.516776 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.516739    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9353cf66-a8b0-4281-86b1-90d2b12dc37a-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.516776 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.516774    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phzl9\" (UniqueName: \"kubernetes.io/projected/9353cf66-a8b0-4281-86b1-90d2b12dc37a-kube-api-access-phzl9\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.625831 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.625798    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"]
Sep 18 20:18:13.634722 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.634665    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x"]
Sep 18 20:18:13.700744 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.700633    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bb2641e-4cb0-4b8a-bca3-61bee7ea2166" path="/var/lib/kubelet/pods/7bb2641e-4cb0-4b8a-bca3-61bee7ea2166/volumes"
Sep 18 20:18:13.701174 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:13.701152    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83f8e21b-264a-4a6f-a823-b6878475b2cd" path="/var/lib/kubelet/pods/83f8e21b-264a-4a6f-a823-b6878475b2cd/volumes"
Sep 18 20:18:14.286462 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:14.286431    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"
Sep 18 20:18:14.286671 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:14.286459    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s" event={"ID":"9353cf66-a8b0-4281-86b1-90d2b12dc37a","Type":"ContainerDied","Data":"8e06958c064d6e563600c12364ba1b96234fbe37646f296e5ec3821f837fab61"}
Sep 18 20:18:14.286671 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:14.286532    2567 scope.go:117] "RemoveContainer" containerID="dcfcc425efd384fdc6e83ad7efb47b3265e7b6213dd1761e231152a036b3dec5"
Sep 18 20:18:14.328087 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:14.328057    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"]
Sep 18 20:18:14.332466 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:14.332433    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-c34fad-7c9ff8f98f-2b29s"]
Sep 18 20:18:15.700154 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:15.700119    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9353cf66-a8b0-4281-86b1-90d2b12dc37a" path="/var/lib/kubelet/pods/9353cf66-a8b0-4281-86b1-90d2b12dc37a/volumes"
Sep 18 20:18:16.075446 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:16.075355    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c34fad-79f6df4c96-m7b4x" podUID="83f8e21b-264a-4a6f-a823-b6878475b2cd" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.48:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 18 20:18:24.290557 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:24.290523    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:18:33.936203 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.936169    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"]
Sep 18 20:18:33.936818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.936695    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83f8e21b-264a-4a6f-a823-b6878475b2cd" containerName="maas-api"
Sep 18 20:18:33.936818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.936715    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="83f8e21b-264a-4a6f-a823-b6878475b2cd" containerName="maas-api"
Sep 18 20:18:33.936818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.936731    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9353cf66-a8b0-4281-86b1-90d2b12dc37a" containerName="payload-processing"
Sep 18 20:18:33.936818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.936737    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9353cf66-a8b0-4281-86b1-90d2b12dc37a" containerName="payload-processing"
Sep 18 20:18:33.937053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.936844    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9353cf66-a8b0-4281-86b1-90d2b12dc37a" containerName="payload-processing"
Sep 18 20:18:33.937053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.936856    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="83f8e21b-264a-4a6f-a823-b6878475b2cd" containerName="maas-api"
Sep 18 20:18:33.940007 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.939979    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:33.942875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.942847    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-c5206e-serving-cert\""
Sep 18 20:18:33.942997 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.942882    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kcjh\""
Sep 18 20:18:33.950997 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:33.950974    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"]
Sep 18 20:18:34.024002 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.023965    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.024184 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.024009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/158d6542-f9cf-4d2a-b1bc-0623419a4895-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.024184 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.024074    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mpb8\" (UniqueName: \"kubernetes.io/projected/158d6542-f9cf-4d2a-b1bc-0623419a4895-kube-api-access-8mpb8\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.024184 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.024096    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.125133 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.125092    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mpb8\" (UniqueName: \"kubernetes.io/projected/158d6542-f9cf-4d2a-b1bc-0623419a4895-kube-api-access-8mpb8\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.125133 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.125140    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.125422 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.125257    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.125422 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.125293    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/158d6542-f9cf-4d2a-b1bc-0623419a4895-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.125422 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:18:34.125324    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-serving-cert: secret "maas-api-e2e-shared-b-w2-c5206e-serving-cert" not found
Sep 18 20:18:34.125422 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:18:34.125407    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-maas-api-tls podName:158d6542-f9cf-4d2a-b1bc-0623419a4895 nodeName:}" failed. No retries permitted until 2026-09-18 20:18:34.625386626 +0000 UTC m=+1295.185970063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-maas-api-tls") pod "maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl" (UID: "158d6542-f9cf-4d2a-b1bc-0623419a4895") : secret "maas-api-e2e-shared-b-w2-c5206e-serving-cert" not found
Sep 18 20:18:34.126032 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.126002    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/158d6542-f9cf-4d2a-b1bc-0623419a4895-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.127841 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.127823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.134453 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.134424    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mpb8\" (UniqueName: \"kubernetes.io/projected/158d6542-f9cf-4d2a-b1bc-0623419a4895-kube-api-access-8mpb8\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.630180 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.630143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.632519 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.632494    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.846331 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.846299    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"]
Sep 18 20:18:34.850278 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.850257    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:18:34.852580 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.852546    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:34.853637 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.853614    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-c5206e\""
Sep 18 20:18:34.853754 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.853738    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-c5206e-dockercfg-wt8fq\""
Sep 18 20:18:34.863292 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.863265    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"]
Sep 18 20:18:34.933994 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.933963    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df0ae129-30f2-42ec-8539-f9d15b880efa-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4\" (UID: \"df0ae129-30f2-42ec-8539-f9d15b880efa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:18:34.934165 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:34.934146    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5kfn\" (UniqueName: \"kubernetes.io/projected/df0ae129-30f2-42ec-8539-f9d15b880efa-kube-api-access-c5kfn\") pod \"payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4\" (UID: \"df0ae129-30f2-42ec-8539-f9d15b880efa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:18:35.035748 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:35.035708    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5kfn\" (UniqueName: \"kubernetes.io/projected/df0ae129-30f2-42ec-8539-f9d15b880efa-kube-api-access-c5kfn\") pod \"payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4\" (UID: \"df0ae129-30f2-42ec-8539-f9d15b880efa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:18:35.036246 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:35.035791    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df0ae129-30f2-42ec-8539-f9d15b880efa-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4\" (UID: \"df0ae129-30f2-42ec-8539-f9d15b880efa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:18:35.036654 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:35.036626    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df0ae129-30f2-42ec-8539-f9d15b880efa-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4\" (UID: \"df0ae129-30f2-42ec-8539-f9d15b880efa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:18:35.045293 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:35.045259    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5kfn\" (UniqueName: \"kubernetes.io/projected/df0ae129-30f2-42ec-8539-f9d15b880efa-kube-api-access-c5kfn\") pod \"payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4\" (UID: \"df0ae129-30f2-42ec-8539-f9d15b880efa\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:18:35.151890 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:35.151815    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"]
Sep 18 20:18:35.154414 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:18:35.154378    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod158d6542_f9cf_4d2a_b1bc_0623419a4895.slice/crio-bfb1fcd1f9a63080e7bb31d8b654e058f1bf3b7106b19fac9227e4de694a1018 WatchSource:0}: Error finding container bfb1fcd1f9a63080e7bb31d8b654e058f1bf3b7106b19fac9227e4de694a1018: Status 404 returned error can't find the container with id bfb1fcd1f9a63080e7bb31d8b654e058f1bf3b7106b19fac9227e4de694a1018
Sep 18 20:18:35.163601 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:35.163576    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:18:35.409026 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:35.408991    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl" event={"ID":"158d6542-f9cf-4d2a-b1bc-0623419a4895","Type":"ContainerStarted","Data":"bfb1fcd1f9a63080e7bb31d8b654e058f1bf3b7106b19fac9227e4de694a1018"}
Sep 18 20:18:35.452540 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:35.452511    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"]
Sep 18 20:18:35.453577 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:18:35.453553    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf0ae129_30f2_42ec_8539_f9d15b880efa.slice/crio-9d9cd40c3418d400c98c1be811382ec82ac2e86f8fd9b020f10547c1a9e6f137 WatchSource:0}: Error finding container 9d9cd40c3418d400c98c1be811382ec82ac2e86f8fd9b020f10547c1a9e6f137: Status 404 returned error can't find the container with id 9d9cd40c3418d400c98c1be811382ec82ac2e86f8fd9b020f10547c1a9e6f137
Sep 18 20:18:36.416020 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:36.415985    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl" event={"ID":"158d6542-f9cf-4d2a-b1bc-0623419a4895","Type":"ContainerStarted","Data":"4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870"}
Sep 18 20:18:36.416471 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:36.416079    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:36.417744 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:36.417722    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4" event={"ID":"df0ae129-30f2-42ec-8539-f9d15b880efa","Type":"ContainerStarted","Data":"6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b"}
Sep 18 20:18:36.417860 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:36.417766    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:18:36.417860 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:36.417789    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4" event={"ID":"df0ae129-30f2-42ec-8539-f9d15b880efa","Type":"ContainerStarted","Data":"9d9cd40c3418d400c98c1be811382ec82ac2e86f8fd9b020f10547c1a9e6f137"}
Sep 18 20:18:36.465122 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:36.465039    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4" podStartSLOduration=2.465018929 podStartE2EDuration="2.465018929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:18:36.464988205 +0000 UTC m=+1297.025571663" watchObservedRunningTime="2026-09-18 20:18:36.465018929 +0000 UTC m=+1297.025602389"
Sep 18 20:18:36.466135 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:36.466092    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl" podStartSLOduration=3.154845124 podStartE2EDuration="3.466076962s" totalImagesPullingTime="311.231838ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:18:36.444108261 +0000 UTC m=+1297.004691719" watchObservedRunningTime="2026-09-18 20:18:36.466076962 +0000 UTC m=+1297.026660421"
Sep 18 20:18:42.430195 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:42.430165    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:18:47.426199 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:18:47.426170    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:19:13.917620 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:13.917526    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"]
Sep 18 20:19:13.922977 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:13.922955    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:13.925618 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:13.925599    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-81f19d-serving-cert\""
Sep 18 20:19:13.939617 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:13.939520    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"]
Sep 18 20:19:14.054702 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.054647    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3536e404-87d3-439d-8b39-2469dcf63406-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.054897 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.054738    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.054897 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.054793    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.054897 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.054824    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp4zt\" (UniqueName: \"kubernetes.io/projected/3536e404-87d3-439d-8b39-2469dcf63406-kube-api-access-xp4zt\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.156219 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.156186    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3536e404-87d3-439d-8b39-2469dcf63406-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.156425 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.156236    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.156425 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.156273    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.156425 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.156294    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp4zt\" (UniqueName: \"kubernetes.io/projected/3536e404-87d3-439d-8b39-2469dcf63406-kube-api-access-xp4zt\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.156600 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:19:14.156422    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-serving-cert: secret "maas-api-e2e-shared-b-w4-81f19d-serving-cert" not found
Sep 18 20:19:14.156600 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:19:14.156501    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-maas-api-tls podName:3536e404-87d3-439d-8b39-2469dcf63406 nodeName:}" failed. No retries permitted until 2026-09-18 20:19:14.656479131 +0000 UTC m=+1335.217062570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-maas-api-tls") pod "maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps" (UID: "3536e404-87d3-439d-8b39-2469dcf63406") : secret "maas-api-e2e-shared-b-w4-81f19d-serving-cert" not found
Sep 18 20:19:14.157212 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.157187    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3536e404-87d3-439d-8b39-2469dcf63406-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.158846 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.158828    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.177425 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.177360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp4zt\" (UniqueName: \"kubernetes.io/projected/3536e404-87d3-439d-8b39-2469dcf63406-kube-api-access-xp4zt\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.662348 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.662306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.664926 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.664895    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.834394 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.834360    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:14.996579 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:14.996498    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"]
Sep 18 20:19:15.000435 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.000419    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:15.003709 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.003677    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w4-81f19d-dockercfg-77mhm\""
Sep 18 20:19:15.005967 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.005947    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w4-81f19d\""
Sep 18 20:19:15.022087 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.022054    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"]
Sep 18 20:19:15.127457 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.127428    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"]
Sep 18 20:19:15.129476 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:19:15.129445    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3536e404_87d3_439d_8b39_2469dcf63406.slice/crio-3316363d461f2ab2654a327886db7ccb6882bfe16bc7effec29b24d8f1acc0a9 WatchSource:0}: Error finding container 3316363d461f2ab2654a327886db7ccb6882bfe16bc7effec29b24d8f1acc0a9: Status 404 returned error can't find the container with id 3316363d461f2ab2654a327886db7ccb6882bfe16bc7effec29b24d8f1acc0a9
Sep 18 20:19:15.168527 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.168492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mchc\" (UniqueName: \"kubernetes.io/projected/9447be53-157e-44da-a486-08b92a76cf33-kube-api-access-4mchc\") pod \"payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz\" (UID: \"9447be53-157e-44da-a486-08b92a76cf33\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:15.168713 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.168550    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9447be53-157e-44da-a486-08b92a76cf33-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz\" (UID: \"9447be53-157e-44da-a486-08b92a76cf33\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:15.270042 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.269938    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mchc\" (UniqueName: \"kubernetes.io/projected/9447be53-157e-44da-a486-08b92a76cf33-kube-api-access-4mchc\") pod \"payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz\" (UID: \"9447be53-157e-44da-a486-08b92a76cf33\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:15.270228 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.270087    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9447be53-157e-44da-a486-08b92a76cf33-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz\" (UID: \"9447be53-157e-44da-a486-08b92a76cf33\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:15.270850 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.270824    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9447be53-157e-44da-a486-08b92a76cf33-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz\" (UID: \"9447be53-157e-44da-a486-08b92a76cf33\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:15.280753 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.280728    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mchc\" (UniqueName: \"kubernetes.io/projected/9447be53-157e-44da-a486-08b92a76cf33-kube-api-access-4mchc\") pod \"payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz\" (UID: \"9447be53-157e-44da-a486-08b92a76cf33\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:15.311351 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.311321    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:15.602204 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.602176    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"]
Sep 18 20:19:15.604352 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:19:15.604330    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9447be53_157e_44da_a486_08b92a76cf33.slice/crio-f09d059e773d9d592eb126978039d94391113f343a37da0cadc8bc0425064b61 WatchSource:0}: Error finding container f09d059e773d9d592eb126978039d94391113f343a37da0cadc8bc0425064b61: Status 404 returned error can't find the container with id f09d059e773d9d592eb126978039d94391113f343a37da0cadc8bc0425064b61
Sep 18 20:19:15.645264 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.645222    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz" event={"ID":"9447be53-157e-44da-a486-08b92a76cf33","Type":"ContainerStarted","Data":"f09d059e773d9d592eb126978039d94391113f343a37da0cadc8bc0425064b61"}
Sep 18 20:19:15.646815 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:15.646785    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps" event={"ID":"3536e404-87d3-439d-8b39-2469dcf63406","Type":"ContainerStarted","Data":"3316363d461f2ab2654a327886db7ccb6882bfe16bc7effec29b24d8f1acc0a9"}
Sep 18 20:19:16.654007 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:16.653965    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz" event={"ID":"9447be53-157e-44da-a486-08b92a76cf33","Type":"ContainerStarted","Data":"e7111723d8ad99e14f57880a425678f4c7c259617697df139568497bb9a9d2ca"}
Sep 18 20:19:16.654457 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:16.654159    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:16.655794 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:16.655767    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps" event={"ID":"3536e404-87d3-439d-8b39-2469dcf63406","Type":"ContainerStarted","Data":"5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b"}
Sep 18 20:19:16.655913 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:16.655829    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:16.675021 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:16.674974    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz" podStartSLOduration=2.674959695 podStartE2EDuration="2.674959695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:19:16.672071671 +0000 UTC m=+1337.232655128" watchObservedRunningTime="2026-09-18 20:19:16.674959695 +0000 UTC m=+1337.235543153"
Sep 18 20:19:16.693343 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:16.693294    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps" podStartSLOduration=2.796890846 podStartE2EDuration="3.693277915s" totalImagesPullingTime="896.387069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:19:16.691195589 +0000 UTC m=+1337.251779044" watchObservedRunningTime="2026-09-18 20:19:16.693277915 +0000 UTC m=+1337.253861371"
Sep 18 20:19:22.239216 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.239181    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"]
Sep 18 20:19:22.243326 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.243303    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.245985 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.245963    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7d49a30d-serving-cert\""
Sep 18 20:19:22.258249 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.258221    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"]
Sep 18 20:19:22.339729 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.339680    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a428e2-3d33-479d-afe4-e59e94f66351-openshift-service-ca\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.339917 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.339805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-maas-api-tls\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.339917 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.339827    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsqgd\" (UniqueName: \"kubernetes.io/projected/36a428e2-3d33-479d-afe4-e59e94f66351-kube-api-access-hsqgd\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.339917 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.339861    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-metrics-certs\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.441089 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.441055    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-maas-api-tls\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.441089 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.441090    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsqgd\" (UniqueName: \"kubernetes.io/projected/36a428e2-3d33-479d-afe4-e59e94f66351-kube-api-access-hsqgd\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.441335 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:19:22.441187    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-serving-cert: secret "maas-api-e2e-mt-7d49a30d-serving-cert" not found
Sep 18 20:19:22.441335 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.441204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-metrics-certs\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.441335 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:19:22.441244    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-maas-api-tls podName:36a428e2-3d33-479d-afe4-e59e94f66351 nodeName:}" failed. No retries permitted until 2026-09-18 20:19:22.941227987 +0000 UTC m=+1343.501811421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-maas-api-tls") pod "maas-api-e2e-mt-7d49a30d-647cff896b-slq6r" (UID: "36a428e2-3d33-479d-afe4-e59e94f66351") : secret "maas-api-e2e-mt-7d49a30d-serving-cert" not found
Sep 18 20:19:22.441506 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.441341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a428e2-3d33-479d-afe4-e59e94f66351-openshift-service-ca\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.441971 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.441951    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a428e2-3d33-479d-afe4-e59e94f66351-openshift-service-ca\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.443637 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.443617    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-metrics-certs\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.453232 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.453204    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsqgd\" (UniqueName: \"kubernetes.io/projected/36a428e2-3d33-479d-afe4-e59e94f66351-kube-api-access-hsqgd\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.666850 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.666818    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:19:22.947345 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.947239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-maas-api-tls\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:22.950114 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:22.950086    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-maas-api-tls\") pod \"maas-api-e2e-mt-7d49a30d-647cff896b-slq6r\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:23.111964 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.111924    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"]
Sep 18 20:19:23.115590 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.115566    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:19:23.118207 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.118180    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-7d49a30d-dockercfg-g5fsj\""
Sep 18 20:19:23.118337 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.118264    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-7d49a30d\""
Sep 18 20:19:23.132373 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.132348    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"]
Sep 18 20:19:23.149676 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.149638    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsfkc\" (UniqueName: \"kubernetes.io/projected/8bbf164f-58fe-427a-a705-4f4e22223388-kube-api-access-jsfkc\") pod \"payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57\" (UID: \"8bbf164f-58fe-427a-a705-4f4e22223388\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:19:23.149910 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.149722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf164f-58fe-427a-a705-4f4e22223388-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57\" (UID: \"8bbf164f-58fe-427a-a705-4f4e22223388\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:19:23.154233 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.154202    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:23.250967 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.250935    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf164f-58fe-427a-a705-4f4e22223388-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57\" (UID: \"8bbf164f-58fe-427a-a705-4f4e22223388\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:19:23.251443 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.251125    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsfkc\" (UniqueName: \"kubernetes.io/projected/8bbf164f-58fe-427a-a705-4f4e22223388-kube-api-access-jsfkc\") pod \"payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57\" (UID: \"8bbf164f-58fe-427a-a705-4f4e22223388\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:19:23.251926 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.251901    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf164f-58fe-427a-a705-4f4e22223388-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57\" (UID: \"8bbf164f-58fe-427a-a705-4f4e22223388\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:19:23.260171 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.260134    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsfkc\" (UniqueName: \"kubernetes.io/projected/8bbf164f-58fe-427a-a705-4f4e22223388-kube-api-access-jsfkc\") pod \"payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57\" (UID: \"8bbf164f-58fe-427a-a705-4f4e22223388\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:19:23.427892 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.427856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:19:23.443718 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.443672    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"]
Sep 18 20:19:23.445197 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:19:23.445165    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a428e2_3d33_479d_afe4_e59e94f66351.slice/crio-edefa1f0c8339ac54bb66eaba490c8e86b656dbf8f4e32839b1b9089cb761afa WatchSource:0}: Error finding container edefa1f0c8339ac54bb66eaba490c8e86b656dbf8f4e32839b1b9089cb761afa: Status 404 returned error can't find the container with id edefa1f0c8339ac54bb66eaba490c8e86b656dbf8f4e32839b1b9089cb761afa
Sep 18 20:19:23.699937 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.699905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r" event={"ID":"36a428e2-3d33-479d-afe4-e59e94f66351","Type":"ContainerStarted","Data":"edefa1f0c8339ac54bb66eaba490c8e86b656dbf8f4e32839b1b9089cb761afa"}
Sep 18 20:19:23.726573 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:23.726543    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"]
Sep 18 20:19:23.729868 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:19:23.729834    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bbf164f_58fe_427a_a705_4f4e22223388.slice/crio-92c6e56cc2b043d463628a392342b813937e5935c8780fb73d20d0678d798bce WatchSource:0}: Error finding container 92c6e56cc2b043d463628a392342b813937e5935c8780fb73d20d0678d798bce: Status 404 returned error can't find the container with id 92c6e56cc2b043d463628a392342b813937e5935c8780fb73d20d0678d798bce
Sep 18 20:19:24.704466 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:24.704426    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57" event={"ID":"8bbf164f-58fe-427a-a705-4f4e22223388","Type":"ContainerStarted","Data":"f3a5271974d1f748437ab5c1d4700539e8d48695ad8d9a7828e61311baf5e591"}
Sep 18 20:19:24.704957 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:24.704481    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57" event={"ID":"8bbf164f-58fe-427a-a705-4f4e22223388","Type":"ContainerStarted","Data":"92c6e56cc2b043d463628a392342b813937e5935c8780fb73d20d0678d798bce"}
Sep 18 20:19:24.704957 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:24.704594    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:19:24.706166 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:24.706124    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r" event={"ID":"36a428e2-3d33-479d-afe4-e59e94f66351","Type":"ContainerStarted","Data":"5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c"}
Sep 18 20:19:24.706366 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:24.706347    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:24.739598 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:24.739549    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57" podStartSLOduration=1.739535083 podStartE2EDuration="1.739535083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:19:24.736319966 +0000 UTC m=+1345.296903422" watchObservedRunningTime="2026-09-18 20:19:24.739535083 +0000 UTC m=+1345.300118517"
Sep 18 20:19:24.767894 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:24.767845    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r" podStartSLOduration=2.43978685 podStartE2EDuration="2.767828797s" totalImagesPullingTime="328.041947ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:19:24.765758693 +0000 UTC m=+1345.326342151" watchObservedRunningTime="2026-09-18 20:19:24.767828797 +0000 UTC m=+1345.328412255"
Sep 18 20:19:27.663792 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:27.663760    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:19:30.717142 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:30.717107    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:19:35.713530 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:19:35.713495    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:20:30.825281 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.825239    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"]
Sep 18 20:20:30.829080 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.829056    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.831998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.831973    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-5284dc\"/\"embedding-w0-5284dc-kserve-self-signed-certs\""
Sep 18 20:20:30.842548 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.842527    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"]
Sep 18 20:20:30.878109 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.878079    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-tmp-dir\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.878266 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.878153    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfzp4\" (UniqueName: \"kubernetes.io/projected/3f8137af-b163-4f00-997f-84b5f2e95dc0-kube-api-access-nfzp4\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.878266 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.878203    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-dshm\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.878363 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.878275    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-home\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.878363 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.878358    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-model-cache\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.878446 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.878418    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f8137af-b163-4f00-997f-84b5f2e95dc0-tls-certs\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.979005 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.978968    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-model-cache\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.979199 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.979052    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f8137af-b163-4f00-997f-84b5f2e95dc0-tls-certs\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.979199 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.979081    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-tmp-dir\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.979199 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.979156    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfzp4\" (UniqueName: \"kubernetes.io/projected/3f8137af-b163-4f00-997f-84b5f2e95dc0-kube-api-access-nfzp4\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.979199 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.979188    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-dshm\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.979441 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.979242    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-home\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.979516 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.979439    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-model-cache\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.979516 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.979493    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-tmp-dir\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.979680 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.979662    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-home\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.981446 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.981421    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-dshm\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.981652 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.981635    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f8137af-b163-4f00-997f-84b5f2e95dc0-tls-certs\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:30.992063 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:30.992040    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfzp4\" (UniqueName: \"kubernetes.io/projected/3f8137af-b163-4f00-997f-84b5f2e95dc0-kube-api-access-nfzp4\") pod \"embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") " pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:31.138356 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:31.138324    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:31.415651 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:31.415617    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"]
Sep 18 20:20:31.418483 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:20:31.418454    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f8137af_b163_4f00_997f_84b5f2e95dc0.slice/crio-0913ce4e04ce7ed47a8abc215109222339aaab71cd80591b3d3e5dc32bcff33b WatchSource:0}: Error finding container 0913ce4e04ce7ed47a8abc215109222339aaab71cd80591b3d3e5dc32bcff33b: Status 404 returned error can't find the container with id 0913ce4e04ce7ed47a8abc215109222339aaab71cd80591b3d3e5dc32bcff33b
Sep 18 20:20:32.094326 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:32.094288    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4" event={"ID":"3f8137af-b163-4f00-997f-84b5f2e95dc0","Type":"ContainerStarted","Data":"b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202"}
Sep 18 20:20:32.094326 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:32.094334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4" event={"ID":"3f8137af-b163-4f00-997f-84b5f2e95dc0","Type":"ContainerStarted","Data":"0913ce4e04ce7ed47a8abc215109222339aaab71cd80591b3d3e5dc32bcff33b"}
Sep 18 20:20:32.114380 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:32.114333    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4" podStartSLOduration=2.114316188 podStartE2EDuration="2.114316188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:20:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:32.113128375 +0000 UTC m=+1412.673711847" watchObservedRunningTime="2026-09-18 20:20:32.114316188 +0000 UTC m=+1412.674899644"
Sep 18 20:20:41.139147 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:41.139114    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:41.139603 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:41.139165    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:41.152302 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:41.152273    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:20:41.164779 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:20:41.164747    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:21:30.576232 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.576198    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"]
Sep 18 20:21:30.579895 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.579878    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.582662 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.582641    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w4-81f19d\"/\"kube-root-ca.crt\""
Sep 18 20:21:30.583664 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.583637    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w4-81f19d\"/\"default-dockercfg-kwxkf\""
Sep 18 20:21:30.583829 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.583704    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w4-81f19d\"/\"openshift-service-ca.crt\""
Sep 18 20:21:30.583829 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.583728    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w4-81f19d\"/\"auth-test-model-81f19d-kserve-self-signed-certs\""
Sep 18 20:21:30.607079 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.607042    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"]
Sep 18 20:21:30.608593 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.608560    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-dshm\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.608777 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.608616    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7xks\" (UniqueName: \"kubernetes.io/projected/29402ff9-6864-4cd2-8b7b-024dec9d07b4-kube-api-access-t7xks\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.608777 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.608723    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-model-cache\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.608914 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.608796    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-home\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.608914 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.608831    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tmp-dir\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.608914 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.608895    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tls-certs\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.709520 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.709483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-dshm\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.709775 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.709541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7xks\" (UniqueName: \"kubernetes.io/projected/29402ff9-6864-4cd2-8b7b-024dec9d07b4-kube-api-access-t7xks\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.709775 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.709573    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-model-cache\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.709775 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.709628    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-home\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.709775 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.709666    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tmp-dir\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.709775 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.709743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tls-certs\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.710130 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.710027    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-model-cache\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.710130 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.710046    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-home\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.710130 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.710077    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tmp-dir\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.711957 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.711924    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-dshm\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.712306 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.712288    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tls-certs\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.718571 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.718545    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7xks\" (UniqueName: \"kubernetes.io/projected/29402ff9-6864-4cd2-8b7b-024dec9d07b4-kube-api-access-t7xks\") pod \"auth-test-model-81f19d-kserve-855577d8c9-65ddk\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") " pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:30.890004 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:30.889961    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:31.159647 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.159553    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"]
Sep 18 20:21:31.159841 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.159771    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r" podUID="36a428e2-3d33-479d-afe4-e59e94f66351" containerName="maas-api" containerID="cri-o://5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c" gracePeriod=30
Sep 18 20:21:31.179943 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.179898    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"]
Sep 18 20:21:31.182650 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:21:31.182601    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29402ff9_6864_4cd2_8b7b_024dec9d07b4.slice/crio-4596bf836ebd1a40e8b308d3bc68019b339def703b1481cdf0de18981d20bb39 WatchSource:0}: Error finding container 4596bf836ebd1a40e8b308d3bc68019b339def703b1481cdf0de18981d20bb39: Status 404 returned error can't find the container with id 4596bf836ebd1a40e8b308d3bc68019b339def703b1481cdf0de18981d20bb39
Sep 18 20:21:31.295299 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.295266    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"]
Sep 18 20:21:31.295878 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.295502    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57" podUID="8bbf164f-58fe-427a-a705-4f4e22223388" containerName="payload-pre-processing" containerID="cri-o://f3a5271974d1f748437ab5c1d4700539e8d48695ad8d9a7828e61311baf5e591" gracePeriod=30
Sep 18 20:21:31.418375 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.418339    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:21:31.448032 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.447792    2567 generic.go:356] "Generic (PLEG): container finished" podID="36a428e2-3d33-479d-afe4-e59e94f66351" containerID="5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c" exitCode=0
Sep 18 20:21:31.448032 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.447927    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r" event={"ID":"36a428e2-3d33-479d-afe4-e59e94f66351","Type":"ContainerDied","Data":"5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c"}
Sep 18 20:21:31.448032 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.447984    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r" event={"ID":"36a428e2-3d33-479d-afe4-e59e94f66351","Type":"ContainerDied","Data":"edefa1f0c8339ac54bb66eaba490c8e86b656dbf8f4e32839b1b9089cb761afa"}
Sep 18 20:21:31.448032 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.448016    2567 scope.go:117] "RemoveContainer" containerID="5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c"
Sep 18 20:21:31.449349 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.448188    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"
Sep 18 20:21:31.452918 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.452738    2567 generic.go:356] "Generic (PLEG): container finished" podID="8bbf164f-58fe-427a-a705-4f4e22223388" containerID="f3a5271974d1f748437ab5c1d4700539e8d48695ad8d9a7828e61311baf5e591" exitCode=0
Sep 18 20:21:31.452918 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.452819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57" event={"ID":"8bbf164f-58fe-427a-a705-4f4e22223388","Type":"ContainerDied","Data":"f3a5271974d1f748437ab5c1d4700539e8d48695ad8d9a7828e61311baf5e591"}
Sep 18 20:21:31.456103 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.455624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk" event={"ID":"29402ff9-6864-4cd2-8b7b-024dec9d07b4","Type":"ContainerStarted","Data":"185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f"}
Sep 18 20:21:31.456103 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.455664    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk" event={"ID":"29402ff9-6864-4cd2-8b7b-024dec9d07b4","Type":"ContainerStarted","Data":"4596bf836ebd1a40e8b308d3bc68019b339def703b1481cdf0de18981d20bb39"}
Sep 18 20:21:31.463555 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.463533    2567 scope.go:117] "RemoveContainer" containerID="5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c"
Sep 18 20:21:31.463950 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:21:31.463925    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c\": container with ID starting with 5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c not found: ID does not exist" containerID="5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c"
Sep 18 20:21:31.464030 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.463953    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c"} err="failed to get container status \"5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c\": rpc error: code = NotFound desc = could not find container \"5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c\": container with ID starting with 5c1b3cd103dc8d5e909973e447fcf0cf5429a60106c20d6fb1e4d7539b9d076c not found: ID does not exist"
Sep 18 20:21:31.480883 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.480813    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk" podStartSLOduration=1.480788934 podStartE2EDuration="1.480788934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:31.477992772 +0000 UTC m=+1472.038576229" watchObservedRunningTime="2026-09-18 20:21:31.480788934 +0000 UTC m=+1472.041372390"
Sep 18 20:21:31.516805 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.516773    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-metrics-certs\") pod \"36a428e2-3d33-479d-afe4-e59e94f66351\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") "
Sep 18 20:21:31.516982 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.516877    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-maas-api-tls\") pod \"36a428e2-3d33-479d-afe4-e59e94f66351\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") "
Sep 18 20:21:31.517065 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.517048    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsqgd\" (UniqueName: \"kubernetes.io/projected/36a428e2-3d33-479d-afe4-e59e94f66351-kube-api-access-hsqgd\") pod \"36a428e2-3d33-479d-afe4-e59e94f66351\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") "
Sep 18 20:21:31.517105 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.517082    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a428e2-3d33-479d-afe4-e59e94f66351-openshift-service-ca\") pod \"36a428e2-3d33-479d-afe4-e59e94f66351\" (UID: \"36a428e2-3d33-479d-afe4-e59e94f66351\") "
Sep 18 20:21:31.517578 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.517541    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a428e2-3d33-479d-afe4-e59e94f66351-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "36a428e2-3d33-479d-afe4-e59e94f66351" (UID: "36a428e2-3d33-479d-afe4-e59e94f66351"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:21:31.518247 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.518223    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a428e2-3d33-479d-afe4-e59e94f66351-openshift-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:21:31.519327 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.519297    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "36a428e2-3d33-479d-afe4-e59e94f66351" (UID: "36a428e2-3d33-479d-afe4-e59e94f66351"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:21:31.519520 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.519496    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "36a428e2-3d33-479d-afe4-e59e94f66351" (UID: "36a428e2-3d33-479d-afe4-e59e94f66351"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:21:31.519630 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.519608    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36a428e2-3d33-479d-afe4-e59e94f66351-kube-api-access-hsqgd" (OuterVolumeSpecName: "kube-api-access-hsqgd") pod "36a428e2-3d33-479d-afe4-e59e94f66351" (UID: "36a428e2-3d33-479d-afe4-e59e94f66351"). InnerVolumeSpecName "kube-api-access-hsqgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:21:31.571803 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.571776    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:21:31.619183 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.619154    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jsfkc\" (UniqueName: \"kubernetes.io/projected/8bbf164f-58fe-427a-a705-4f4e22223388-kube-api-access-jsfkc\") pod \"8bbf164f-58fe-427a-a705-4f4e22223388\" (UID: \"8bbf164f-58fe-427a-a705-4f4e22223388\") "
Sep 18 20:21:31.619183 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.619192    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf164f-58fe-427a-a705-4f4e22223388-plugins-config-volume\") pod \"8bbf164f-58fe-427a-a705-4f4e22223388\" (UID: \"8bbf164f-58fe-427a-a705-4f4e22223388\") "
Sep 18 20:21:31.619604 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.619462    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsqgd\" (UniqueName: \"kubernetes.io/projected/36a428e2-3d33-479d-afe4-e59e94f66351-kube-api-access-hsqgd\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:21:31.619604 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.619472    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:21:31.619604 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.619482    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36a428e2-3d33-479d-afe4-e59e94f66351-maas-api-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:21:31.619722 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.619626    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bbf164f-58fe-427a-a705-4f4e22223388-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8bbf164f-58fe-427a-a705-4f4e22223388" (UID: "8bbf164f-58fe-427a-a705-4f4e22223388"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:21:31.621473 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.621444    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bbf164f-58fe-427a-a705-4f4e22223388-kube-api-access-jsfkc" (OuterVolumeSpecName: "kube-api-access-jsfkc") pod "8bbf164f-58fe-427a-a705-4f4e22223388" (UID: "8bbf164f-58fe-427a-a705-4f4e22223388"). InnerVolumeSpecName "kube-api-access-jsfkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:21:31.720633 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.720546    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsfkc\" (UniqueName: \"kubernetes.io/projected/8bbf164f-58fe-427a-a705-4f4e22223388-kube-api-access-jsfkc\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:21:31.720633 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.720578    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8bbf164f-58fe-427a-a705-4f4e22223388-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:21:31.770579 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.770541    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"]
Sep 18 20:21:31.772731 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:31.772706    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7d49a30d-647cff896b-slq6r"]
Sep 18 20:21:32.464151 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:32.464119    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"
Sep 18 20:21:32.464333 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:32.464156    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57" event={"ID":"8bbf164f-58fe-427a-a705-4f4e22223388","Type":"ContainerDied","Data":"92c6e56cc2b043d463628a392342b813937e5935c8780fb73d20d0678d798bce"}
Sep 18 20:21:32.464333 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:32.464255    2567 scope.go:117] "RemoveContainer" containerID="f3a5271974d1f748437ab5c1d4700539e8d48695ad8d9a7828e61311baf5e591"
Sep 18 20:21:32.482557 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:32.482531    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"]
Sep 18 20:21:32.488772 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:32.488743    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-7d49a30d-666789878-rfx57"]
Sep 18 20:21:33.700326 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:33.700293    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36a428e2-3d33-479d-afe4-e59e94f66351" path="/var/lib/kubelet/pods/36a428e2-3d33-479d-afe4-e59e94f66351/volumes"
Sep 18 20:21:33.700806 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:33.700788    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bbf164f-58fe-427a-a705-4f4e22223388" path="/var/lib/kubelet/pods/8bbf164f-58fe-427a-a705-4f4e22223388/volumes"
Sep 18 20:21:40.890647 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:40.890608    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:40.891176 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:40.890715    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:40.903631 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:40.903607    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:21:41.528053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:21:41.528025    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:22:01.758063 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:01.758031    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:22:01.763852 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:01.763830    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:22:01.862157 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:01.862124    2567 scope.go:117] "RemoveContainer" containerID="1e889c2f16fea475a42644660665efc70534a71a4ef26dbf00bd3e3d7a7a569a"
Sep 18 20:22:04.050112 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:04.050072    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"]
Sep 18 20:22:04.065669 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:04.065632    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"]
Sep 18 20:22:14.913854 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:14.913763    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"]
Sep 18 20:22:14.914357 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:14.914032    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm" podUID="e46fa62e-d8dd-4d48-8091-b5eda92236a3" containerName="payload-pre-processing" containerID="cri-o://dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef" gracePeriod=30
Sep 18 20:22:15.172257 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.172187    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:22:15.277366 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.277322    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e46fa62e-d8dd-4d48-8091-b5eda92236a3-plugins-config-volume\") pod \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\" (UID: \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\") "
Sep 18 20:22:15.277542 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.277384    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjttx\" (UniqueName: \"kubernetes.io/projected/e46fa62e-d8dd-4d48-8091-b5eda92236a3-kube-api-access-hjttx\") pod \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\" (UID: \"e46fa62e-d8dd-4d48-8091-b5eda92236a3\") "
Sep 18 20:22:15.277855 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.277829    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e46fa62e-d8dd-4d48-8091-b5eda92236a3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e46fa62e-d8dd-4d48-8091-b5eda92236a3" (UID: "e46fa62e-d8dd-4d48-8091-b5eda92236a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:15.279586 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.279559    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e46fa62e-d8dd-4d48-8091-b5eda92236a3-kube-api-access-hjttx" (OuterVolumeSpecName: "kube-api-access-hjttx") pod "e46fa62e-d8dd-4d48-8091-b5eda92236a3" (UID: "e46fa62e-d8dd-4d48-8091-b5eda92236a3"). InnerVolumeSpecName "kube-api-access-hjttx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:15.379158 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.379119    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e46fa62e-d8dd-4d48-8091-b5eda92236a3-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:15.379158 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.379149    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjttx\" (UniqueName: \"kubernetes.io/projected/e46fa62e-d8dd-4d48-8091-b5eda92236a3-kube-api-access-hjttx\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:15.709523 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.709494    2567 generic.go:356] "Generic (PLEG): container finished" podID="e46fa62e-d8dd-4d48-8091-b5eda92236a3" containerID="dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef" exitCode=0
Sep 18 20:22:15.709656 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.709551    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"
Sep 18 20:22:15.709656 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.709588    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm" event={"ID":"e46fa62e-d8dd-4d48-8091-b5eda92236a3","Type":"ContainerDied","Data":"dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef"}
Sep 18 20:22:15.709656 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.709637    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm" event={"ID":"e46fa62e-d8dd-4d48-8091-b5eda92236a3","Type":"ContainerDied","Data":"d93c25cdf71921b4d2c167d00f26c9afd60bea4449e77fe360708e3b9e503156"}
Sep 18 20:22:15.709863 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.709664    2567 scope.go:117] "RemoveContainer" containerID="dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef"
Sep 18 20:22:15.719223 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.719204    2567 scope.go:117] "RemoveContainer" containerID="dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef"
Sep 18 20:22:15.719522 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:22:15.719500    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef\": container with ID starting with dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef not found: ID does not exist" containerID="dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef"
Sep 18 20:22:15.719600 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.719539    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef"} err="failed to get container status \"dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef\": rpc error: code = NotFound desc = could not find container \"dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef\": container with ID starting with dd3079363e3cd8c4dc659646240905c3d569cc04c1402c2e935be5ee94087aef not found: ID does not exist"
Sep 18 20:22:15.746937 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.746898    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"]
Sep 18 20:22:15.768849 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:15.768806    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-5284dc-5858845794-wblgm"]
Sep 18 20:22:17.700489 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:17.700448    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e46fa62e-d8dd-4d48-8091-b5eda92236a3" path="/var/lib/kubelet/pods/e46fa62e-d8dd-4d48-8091-b5eda92236a3/volumes"
Sep 18 20:22:19.088275 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.088230    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4" podUID="3f8137af-b163-4f00-997f-84b5f2e95dc0" containerName="main" containerID="cri-o://b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202" gracePeriod=45
Sep 18 20:22:19.353541 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.353510    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:22:19.424292 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.424260    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-dshm\") pod \"3f8137af-b163-4f00-997f-84b5f2e95dc0\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") "
Sep 18 20:22:19.424292 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.424294    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f8137af-b163-4f00-997f-84b5f2e95dc0-tls-certs\") pod \"3f8137af-b163-4f00-997f-84b5f2e95dc0\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") "
Sep 18 20:22:19.424566 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.424360    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-tmp-dir\") pod \"3f8137af-b163-4f00-997f-84b5f2e95dc0\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") "
Sep 18 20:22:19.424566 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.424381    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-model-cache\") pod \"3f8137af-b163-4f00-997f-84b5f2e95dc0\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") "
Sep 18 20:22:19.424566 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.424411    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-home\") pod \"3f8137af-b163-4f00-997f-84b5f2e95dc0\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") "
Sep 18 20:22:19.424566 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.424455    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfzp4\" (UniqueName: \"kubernetes.io/projected/3f8137af-b163-4f00-997f-84b5f2e95dc0-kube-api-access-nfzp4\") pod \"3f8137af-b163-4f00-997f-84b5f2e95dc0\" (UID: \"3f8137af-b163-4f00-997f-84b5f2e95dc0\") "
Sep 18 20:22:19.425327 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.425292    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-home" (OuterVolumeSpecName: "home") pod "3f8137af-b163-4f00-997f-84b5f2e95dc0" (UID: "3f8137af-b163-4f00-997f-84b5f2e95dc0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:19.425454 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.425359    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3f8137af-b163-4f00-997f-84b5f2e95dc0" (UID: "3f8137af-b163-4f00-997f-84b5f2e95dc0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:19.425454 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.425442    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-model-cache" (OuterVolumeSpecName: "model-cache") pod "3f8137af-b163-4f00-997f-84b5f2e95dc0" (UID: "3f8137af-b163-4f00-997f-84b5f2e95dc0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:19.427465 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.427439    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-dshm" (OuterVolumeSpecName: "dshm") pod "3f8137af-b163-4f00-997f-84b5f2e95dc0" (UID: "3f8137af-b163-4f00-997f-84b5f2e95dc0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:19.427771 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.427745    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f8137af-b163-4f00-997f-84b5f2e95dc0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3f8137af-b163-4f00-997f-84b5f2e95dc0" (UID: "3f8137af-b163-4f00-997f-84b5f2e95dc0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:19.427771 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.427761    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f8137af-b163-4f00-997f-84b5f2e95dc0-kube-api-access-nfzp4" (OuterVolumeSpecName: "kube-api-access-nfzp4") pod "3f8137af-b163-4f00-997f-84b5f2e95dc0" (UID: "3f8137af-b163-4f00-997f-84b5f2e95dc0"). InnerVolumeSpecName "kube-api-access-nfzp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:19.525727 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.525677    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-dshm\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:19.525727 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.525720    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f8137af-b163-4f00-997f-84b5f2e95dc0-tls-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:19.525727 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.525729    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-tmp-dir\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:19.525727 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.525737    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-model-cache\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:19.526008 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.525745    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f8137af-b163-4f00-997f-84b5f2e95dc0-home\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:19.526008 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.525755    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfzp4\" (UniqueName: \"kubernetes.io/projected/3f8137af-b163-4f00-997f-84b5f2e95dc0-kube-api-access-nfzp4\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:19.666820 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.666733    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq" podUID="4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" containerName="main" containerID="cri-o://417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a" gracePeriod=45
Sep 18 20:22:19.734949 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.734915    2567 generic.go:356] "Generic (PLEG): container finished" podID="3f8137af-b163-4f00-997f-84b5f2e95dc0" containerID="b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202" exitCode=0
Sep 18 20:22:19.735152 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.735022    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"
Sep 18 20:22:19.735152 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.735054    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4" event={"ID":"3f8137af-b163-4f00-997f-84b5f2e95dc0","Type":"ContainerDied","Data":"b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202"}
Sep 18 20:22:19.735152 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.735116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4" event={"ID":"3f8137af-b163-4f00-997f-84b5f2e95dc0","Type":"ContainerDied","Data":"0913ce4e04ce7ed47a8abc215109222339aaab71cd80591b3d3e5dc32bcff33b"}
Sep 18 20:22:19.735152 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.735139    2567 scope.go:117] "RemoveContainer" containerID="b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202"
Sep 18 20:22:19.769272 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.769212    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"]
Sep 18 20:22:19.772716 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.772679    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/embedding-w0-5284dc-kserve-5cf4979dc8-jmzt4"]
Sep 18 20:22:19.778484 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.778461    2567 scope.go:117] "RemoveContainer" containerID="b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202"
Sep 18 20:22:19.778945 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:22:19.778926    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202\": container with ID starting with b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202 not found: ID does not exist" containerID="b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202"
Sep 18 20:22:19.779023 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.778955    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202"} err="failed to get container status \"b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202\": rpc error: code = NotFound desc = could not find container \"b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202\": container with ID starting with b561372d3787cb19b29476e91589d4f20477a549c99a02118e392afb3d469202 not found: ID does not exist"
Sep 18 20:22:19.912232 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:19.912208    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:22:20.031337 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.031238    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tls-certs\") pod \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") "
Sep 18 20:22:20.031337 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.031328    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-model-cache\") pod \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") "
Sep 18 20:22:20.031540 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.031362    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-dshm\") pod \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") "
Sep 18 20:22:20.031540 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.031401    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2kv2\" (UniqueName: \"kubernetes.io/projected/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-kube-api-access-f2kv2\") pod \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") "
Sep 18 20:22:20.031540 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.031455    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tmp-dir\") pod \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") "
Sep 18 20:22:20.031540 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.031504    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-home\") pod \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\" (UID: \"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6\") "
Sep 18 20:22:20.031745 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.031604    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-model-cache" (OuterVolumeSpecName: "model-cache") pod "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" (UID: "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:20.031825 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.031803    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" (UID: "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:20.031944 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.031919    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-home" (OuterVolumeSpecName: "home") pod "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" (UID: "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:20.032043 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.032024    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tmp-dir\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.032085 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.032050    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-home\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.032085 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.032065    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-model-cache\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.033611 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.033587    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" (UID: "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:20.033713 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.033619    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-kube-api-access-f2kv2" (OuterVolumeSpecName: "kube-api-access-f2kv2") pod "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" (UID: "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6"). InnerVolumeSpecName "kube-api-access-f2kv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:20.033713 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.033639    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-dshm" (OuterVolumeSpecName: "dshm") pod "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" (UID: "4087693a-d020-4fa8-a6f5-5fa2f56ee2a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:20.133005 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.132957    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-dshm\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.133005 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.133004    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2kv2\" (UniqueName: \"kubernetes.io/projected/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-kube-api-access-f2kv2\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.133568 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.133021    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6-tls-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.749886 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.749852    2567 generic.go:356] "Generic (PLEG): container finished" podID="4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" containerID="417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a" exitCode=0
Sep 18 20:22:20.750077 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.749907    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"
Sep 18 20:22:20.750077 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.749933    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq" event={"ID":"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6","Type":"ContainerDied","Data":"417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a"}
Sep 18 20:22:20.750077 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.749971    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq" event={"ID":"4087693a-d020-4fa8-a6f5-5fa2f56ee2a6","Type":"ContainerDied","Data":"f8bef1db502428565290d499da057d243ff518ab351b72f80947ba91f2ff5274"}
Sep 18 20:22:20.750077 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.749991    2567 scope.go:117] "RemoveContainer" containerID="417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a"
Sep 18 20:22:20.759157 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.759138    2567 scope.go:117] "RemoveContainer" containerID="417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a"
Sep 18 20:22:20.759438 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:22:20.759419    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a\": container with ID starting with 417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a not found: ID does not exist" containerID="417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a"
Sep 18 20:22:20.759500 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.759451    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a"} err="failed to get container status \"417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a\": rpc error: code = NotFound desc = could not find container \"417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a\": container with ID starting with 417c397be07c4e316561501e905ca34c2b55bf88eb6f8a2f1f6a7c7ba416be0a not found: ID does not exist"
Sep 18 20:22:20.772851 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.772823    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"]
Sep 18 20:22:20.776065 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:20.776044    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-5284dc/distinct-2-w0-5284dc-kserve-76bd7bdf7d-7jnxq"]
Sep 18 20:22:21.700080 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:21.700047    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f8137af-b163-4f00-997f-84b5f2e95dc0" path="/var/lib/kubelet/pods/3f8137af-b163-4f00-997f-84b5f2e95dc0/volumes"
Sep 18 20:22:21.700486 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:22:21.700472    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" path="/var/lib/kubelet/pods/4087693a-d020-4fa8-a6f5-5fa2f56ee2a6/volumes"
Sep 18 20:24:03.817973 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.817892    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"]
Sep 18 20:24:03.818726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818579    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bbf164f-58fe-427a-a705-4f4e22223388" containerName="payload-pre-processing"
Sep 18 20:24:03.818726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818603    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bbf164f-58fe-427a-a705-4f4e22223388" containerName="payload-pre-processing"
Sep 18 20:24:03.818726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818619    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f8137af-b163-4f00-997f-84b5f2e95dc0" containerName="main"
Sep 18 20:24:03.818726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818628    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f8137af-b163-4f00-997f-84b5f2e95dc0" containerName="main"
Sep 18 20:24:03.818726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818644    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e46fa62e-d8dd-4d48-8091-b5eda92236a3" containerName="payload-pre-processing"
Sep 18 20:24:03.818726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818653    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e46fa62e-d8dd-4d48-8091-b5eda92236a3" containerName="payload-pre-processing"
Sep 18 20:24:03.818726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818664    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" containerName="main"
Sep 18 20:24:03.818726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818672    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" containerName="main"
Sep 18 20:24:03.819210 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818745    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a428e2-3d33-479d-afe4-e59e94f66351" containerName="maas-api"
Sep 18 20:24:03.819210 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.818755    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a428e2-3d33-479d-afe4-e59e94f66351" containerName="maas-api"
Sep 18 20:24:03.819531 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.819508    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f8137af-b163-4f00-997f-84b5f2e95dc0" containerName="main"
Sep 18 20:24:03.819721 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.819697    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e46fa62e-d8dd-4d48-8091-b5eda92236a3" containerName="payload-pre-processing"
Sep 18 20:24:03.819721 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.819719    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4087693a-d020-4fa8-a6f5-5fa2f56ee2a6" containerName="main"
Sep 18 20:24:03.819875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.819730    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a428e2-3d33-479d-afe4-e59e94f66351" containerName="maas-api"
Sep 18 20:24:03.819875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.819743    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bbf164f-58fe-427a-a705-4f4e22223388" containerName="payload-pre-processing"
Sep 18 20:24:03.822959 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.822941    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:03.825332 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.825309    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-4cd712-serving-cert\""
Sep 18 20:24:03.834993 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.834966    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"]
Sep 18 20:24:03.998071 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.998032    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-maas-api-tls\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:03.998291 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.998076    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-openshift-service-ca\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:03.998291 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.998190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdnrj\" (UniqueName: \"kubernetes.io/projected/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-kube-api-access-fdnrj\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:03.998291 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:03.998246    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-metrics-certs\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.099231 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.099189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdnrj\" (UniqueName: \"kubernetes.io/projected/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-kube-api-access-fdnrj\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.099431 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.099246    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-metrics-certs\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.099431 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.099327    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-maas-api-tls\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.099431 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.099354    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-openshift-service-ca\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.100184 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.100148    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-openshift-service-ca\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.101949 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.101926    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-metrics-certs\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.102075 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.101964    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-maas-api-tls\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.108206 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.108183    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdnrj\" (UniqueName: \"kubernetes.io/projected/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-kube-api-access-fdnrj\") pod \"maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.137282 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.137255    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:04.426960 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.426904    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"]
Sep 18 20:24:04.428670 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:24:04.428638    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c899ff_fd54_44f0_9c0c_ec9a71a221a9.slice/crio-977658219e4ff122878c49818e5d5cc5768c708c7693d2ee9393af448ed6d931 WatchSource:0}: Error finding container 977658219e4ff122878c49818e5d5cc5768c708c7693d2ee9393af448ed6d931: Status 404 returned error can't find the container with id 977658219e4ff122878c49818e5d5cc5768c708c7693d2ee9393af448ed6d931
Sep 18 20:24:04.430450 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.430427    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:24:04.543936 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.543899    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"]
Sep 18 20:24:04.548762 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.548744    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:04.551200 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.551176    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-4cd712-dockercfg-5vclc\""
Sep 18 20:24:04.551572 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.551540    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-4cd712\""
Sep 18 20:24:04.560810 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.560780    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"]
Sep 18 20:24:04.708133 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.708037    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsmdc\" (UniqueName: \"kubernetes.io/projected/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-kube-api-access-rsmdc\") pod \"payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9\" (UID: \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\") " pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:04.708291 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.708246    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9\" (UID: \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\") " pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:04.809203 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.809162    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9\" (UID: \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\") " pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:04.809410 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.809245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsmdc\" (UniqueName: \"kubernetes.io/projected/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-kube-api-access-rsmdc\") pod \"payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9\" (UID: \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\") " pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:04.810044 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.810017    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9\" (UID: \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\") " pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:04.817663 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.817638    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsmdc\" (UniqueName: \"kubernetes.io/projected/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-kube-api-access-rsmdc\") pod \"payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9\" (UID: \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\") " pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:04.861345 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:04.861311    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:05.137096 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:05.137064    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"]
Sep 18 20:24:05.138263 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:24:05.138235    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d8dedf_81c5_4ad4_ac1f_dac2c3a9c42e.slice/crio-35f2672cccb02a81fffb571eeb49d145ffb2e900d9d3fee789c06bda7302429e WatchSource:0}: Error finding container 35f2672cccb02a81fffb571eeb49d145ffb2e900d9d3fee789c06bda7302429e: Status 404 returned error can't find the container with id 35f2672cccb02a81fffb571eeb49d145ffb2e900d9d3fee789c06bda7302429e
Sep 18 20:24:05.330791 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:05.330752    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9" event={"ID":"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e","Type":"ContainerStarted","Data":"4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b"}
Sep 18 20:24:05.330974 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:05.330812    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9" event={"ID":"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e","Type":"ContainerStarted","Data":"35f2672cccb02a81fffb571eeb49d145ffb2e900d9d3fee789c06bda7302429e"}
Sep 18 20:24:05.330974 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:05.330940    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:05.332416 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:05.332390    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f" event={"ID":"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9","Type":"ContainerStarted","Data":"6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d"}
Sep 18 20:24:05.332536 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:05.332430    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f" event={"ID":"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9","Type":"ContainerStarted","Data":"977658219e4ff122878c49818e5d5cc5768c708c7693d2ee9393af448ed6d931"}
Sep 18 20:24:05.332536 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:05.332450    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:05.353293 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:05.353232    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9" podStartSLOduration=1.353213437 podStartE2EDuration="1.353213437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:24:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:24:05.351349042 +0000 UTC m=+1625.911932502" watchObservedRunningTime="2026-09-18 20:24:05.353213437 +0000 UTC m=+1625.913796897"
Sep 18 20:24:05.392456 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:05.392362    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f" podStartSLOduration=1.95775525 podStartE2EDuration="2.392343829s" totalImagesPullingTime="434.588579ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:24:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:24:05.391490075 +0000 UTC m=+1625.952073532" watchObservedRunningTime="2026-09-18 20:24:05.392343829 +0000 UTC m=+1625.952927287"
Sep 18 20:24:11.343565 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:11.343528    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:24:16.340135 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:16.340101    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:24:34.606550 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:34.606509    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"]
Sep 18 20:24:49.626807 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.626743    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk" podUID="29402ff9-6864-4cd2-8b7b-024dec9d07b4" containerName="main" containerID="cri-o://185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f" gracePeriod=45
Sep 18 20:24:49.889726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.889649    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:24:49.993572 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.993542    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tls-certs\") pod \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") "
Sep 18 20:24:49.993766 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.993612    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-model-cache\") pod \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") "
Sep 18 20:24:49.993766 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.993680    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-home\") pod \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") "
Sep 18 20:24:49.993766 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.993742    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7xks\" (UniqueName: \"kubernetes.io/projected/29402ff9-6864-4cd2-8b7b-024dec9d07b4-kube-api-access-t7xks\") pod \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") "
Sep 18 20:24:49.993886 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.993845    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-dshm\") pod \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") "
Sep 18 20:24:49.993939 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.993910    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tmp-dir\") pod \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\" (UID: \"29402ff9-6864-4cd2-8b7b-024dec9d07b4\") "
Sep 18 20:24:49.993999 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.993984    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-model-cache" (OuterVolumeSpecName: "model-cache") pod "29402ff9-6864-4cd2-8b7b-024dec9d07b4" (UID: "29402ff9-6864-4cd2-8b7b-024dec9d07b4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:49.994062 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.994021    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-home" (OuterVolumeSpecName: "home") pod "29402ff9-6864-4cd2-8b7b-024dec9d07b4" (UID: "29402ff9-6864-4cd2-8b7b-024dec9d07b4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:49.994184 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.994162    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29402ff9-6864-4cd2-8b7b-024dec9d07b4" (UID: "29402ff9-6864-4cd2-8b7b-024dec9d07b4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:49.994325 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.994291    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tmp-dir\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:24:49.994325 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.994306    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-model-cache\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:24:49.994325 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.994315    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-home\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:24:49.995833 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.995808    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29402ff9-6864-4cd2-8b7b-024dec9d07b4" (UID: "29402ff9-6864-4cd2-8b7b-024dec9d07b4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:24:49.995929 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.995904    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29402ff9-6864-4cd2-8b7b-024dec9d07b4-kube-api-access-t7xks" (OuterVolumeSpecName: "kube-api-access-t7xks") pod "29402ff9-6864-4cd2-8b7b-024dec9d07b4" (UID: "29402ff9-6864-4cd2-8b7b-024dec9d07b4"). InnerVolumeSpecName "kube-api-access-t7xks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:24:49.996013 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:49.995989    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-dshm" (OuterVolumeSpecName: "dshm") pod "29402ff9-6864-4cd2-8b7b-024dec9d07b4" (UID: "29402ff9-6864-4cd2-8b7b-024dec9d07b4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:50.095115 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.095070    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7xks\" (UniqueName: \"kubernetes.io/projected/29402ff9-6864-4cd2-8b7b-024dec9d07b4-kube-api-access-t7xks\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:24:50.095115 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.095108    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29402ff9-6864-4cd2-8b7b-024dec9d07b4-dshm\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:24:50.095115 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.095122    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29402ff9-6864-4cd2-8b7b-024dec9d07b4-tls-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:24:50.590846 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.590768    2567 generic.go:356] "Generic (PLEG): container finished" podID="29402ff9-6864-4cd2-8b7b-024dec9d07b4" containerID="185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f" exitCode=0
Sep 18 20:24:50.591080 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.590854    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk" event={"ID":"29402ff9-6864-4cd2-8b7b-024dec9d07b4","Type":"ContainerDied","Data":"185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f"}
Sep 18 20:24:50.591080 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.590915    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk" event={"ID":"29402ff9-6864-4cd2-8b7b-024dec9d07b4","Type":"ContainerDied","Data":"4596bf836ebd1a40e8b308d3bc68019b339def703b1481cdf0de18981d20bb39"}
Sep 18 20:24:50.591080 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.590944    2567 scope.go:117] "RemoveContainer" containerID="185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f"
Sep 18 20:24:50.591421 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.591396    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"
Sep 18 20:24:50.601482 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.601462    2567 scope.go:117] "RemoveContainer" containerID="185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f"
Sep 18 20:24:50.601805 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:24:50.601784    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f\": container with ID starting with 185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f not found: ID does not exist" containerID="185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f"
Sep 18 20:24:50.601878 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.601811    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f"} err="failed to get container status \"185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f\": rpc error: code = NotFound desc = could not find container \"185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f\": container with ID starting with 185f6e3c2c0213aa1e69af09ed9b23afefd3ffafe2d24f6f8e1f2d887424313f not found: ID does not exist"
Sep 18 20:24:50.615356 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.615328    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"]
Sep 18 20:24:50.620201 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:50.620173    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w4-81f19d/auth-test-model-81f19d-kserve-855577d8c9-65ddk"]
Sep 18 20:24:51.699911 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:24:51.699873    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29402ff9-6864-4cd2-8b7b-024dec9d07b4" path="/var/lib/kubelet/pods/29402ff9-6864-4cd2-8b7b-024dec9d07b4/volumes"
Sep 18 20:25:11.952723 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:11.952667    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"]
Sep 18 20:25:11.953306 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:11.952923    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f" podUID="b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" containerName="maas-api" containerID="cri-o://6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d" gracePeriod=30
Sep 18 20:25:12.012704 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.012647    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"]
Sep 18 20:25:12.013020 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.012957    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9" podUID="b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e" containerName="payload-processing" containerID="cri-o://4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b" gracePeriod=30
Sep 18 20:25:12.293589 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.293564    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:25:12.296960 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.296938    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:25:12.332438 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.329967    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-maas-api-tls\") pod \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") "
Sep 18 20:25:12.332438 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.330050    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-openshift-service-ca\") pod \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") "
Sep 18 20:25:12.332438 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.330089    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdnrj\" (UniqueName: \"kubernetes.io/projected/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-kube-api-access-fdnrj\") pod \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") "
Sep 18 20:25:12.332438 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.330125    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-plugins-config-volume\") pod \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\" (UID: \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\") "
Sep 18 20:25:12.332438 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.330760    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsmdc\" (UniqueName: \"kubernetes.io/projected/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-kube-api-access-rsmdc\") pod \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\" (UID: \"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e\") "
Sep 18 20:25:12.332438 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.330795    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-metrics-certs\") pod \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\" (UID: \"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9\") "
Sep 18 20:25:12.334063 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.333608    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" (UID: "b9c899ff-fd54-44f0-9c0c-ec9a71a221a9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:25:12.334063 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.334017    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" (UID: "b9c899ff-fd54-44f0-9c0c-ec9a71a221a9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:25:12.334434 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.334399    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e" (UID: "b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:25:12.339314 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.339277    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-kube-api-access-rsmdc" (OuterVolumeSpecName: "kube-api-access-rsmdc") pod "b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e" (UID: "b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e"). InnerVolumeSpecName "kube-api-access-rsmdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:25:12.340542 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.340516    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" (UID: "b9c899ff-fd54-44f0-9c0c-ec9a71a221a9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:25:12.340818 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.340799    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-kube-api-access-fdnrj" (OuterVolumeSpecName: "kube-api-access-fdnrj") pod "b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" (UID: "b9c899ff-fd54-44f0-9c0c-ec9a71a221a9"). InnerVolumeSpecName "kube-api-access-fdnrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:25:12.432646 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.432613    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsmdc\" (UniqueName: \"kubernetes.io/projected/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-kube-api-access-rsmdc\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:12.432646 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.432641    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:12.432646 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.432649    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-maas-api-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:12.432963 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.432658    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-openshift-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:12.432963 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.432668    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdnrj\" (UniqueName: \"kubernetes.io/projected/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9-kube-api-access-fdnrj\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:12.432963 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.432676    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:12.712148 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.712117    2567 generic.go:356] "Generic (PLEG): container finished" podID="b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e" containerID="4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b" exitCode=0
Sep 18 20:25:12.712148 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.712147    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"
Sep 18 20:25:12.712368 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.712199    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9" event={"ID":"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e","Type":"ContainerDied","Data":"4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b"}
Sep 18 20:25:12.712368 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.712248    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9" event={"ID":"b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e","Type":"ContainerDied","Data":"35f2672cccb02a81fffb571eeb49d145ffb2e900d9d3fee789c06bda7302429e"}
Sep 18 20:25:12.712368 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.712274    2567 scope.go:117] "RemoveContainer" containerID="4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b"
Sep 18 20:25:12.713669 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.713606    2567 generic.go:356] "Generic (PLEG): container finished" podID="b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" containerID="6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d" exitCode=0
Sep 18 20:25:12.713799 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.713710    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"
Sep 18 20:25:12.713799 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.713713    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f" event={"ID":"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9","Type":"ContainerDied","Data":"6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d"}
Sep 18 20:25:12.713906 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.713834    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f" event={"ID":"b9c899ff-fd54-44f0-9c0c-ec9a71a221a9","Type":"ContainerDied","Data":"977658219e4ff122878c49818e5d5cc5768c708c7693d2ee9393af448ed6d931"}
Sep 18 20:25:12.721605 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.721453    2567 scope.go:117] "RemoveContainer" containerID="4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b"
Sep 18 20:25:12.721841 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:25:12.721802    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b\": container with ID starting with 4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b not found: ID does not exist" containerID="4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b"
Sep 18 20:25:12.721948 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.721856    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b"} err="failed to get container status \"4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b\": rpc error: code = NotFound desc = could not find container \"4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b\": container with ID starting with 4d663c35053444653eed330df15a917b24d296a9aedf6c574ec31e5448c7671b not found: ID does not exist"
Sep 18 20:25:12.721948 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.721879    2567 scope.go:117] "RemoveContainer" containerID="6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d"
Sep 18 20:25:12.730567 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.730551    2567 scope.go:117] "RemoveContainer" containerID="6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d"
Sep 18 20:25:12.730849 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:25:12.730829    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d\": container with ID starting with 6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d not found: ID does not exist" containerID="6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d"
Sep 18 20:25:12.730920 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.730856    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d"} err="failed to get container status \"6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d\": rpc error: code = NotFound desc = could not find container \"6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d\": container with ID starting with 6421c23b38d38db0c7bd03b3dc80094ed6e44a09b61f9bcea271e4bfed9ecd5d not found: ID does not exist"
Sep 18 20:25:12.738976 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.738947    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"]
Sep 18 20:25:12.743066 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.743047    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4cd712-7cf6dff569-zn48f"]
Sep 18 20:25:12.753390 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.753362    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"]
Sep 18 20:25:12.757815 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:12.757796    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-4cd712-6bf66cbb55-jv6s9"]
Sep 18 20:25:13.699873 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:13.699843    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e" path="/var/lib/kubelet/pods/b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e/volumes"
Sep 18 20:25:13.700292 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:13.700277    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" path="/var/lib/kubelet/pods/b9c899ff-fd54-44f0-9c0c-ec9a71a221a9/volumes"
Sep 18 20:25:30.512178 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.512134    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"]
Sep 18 20:25:30.512635 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.512420    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d" podUID="8c528839-a6fc-4085-b779-87826bb6588f" containerName="payload-processing" containerID="cri-o://08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9" gracePeriod=30
Sep 18 20:25:30.771153 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.771097    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:25:30.818190 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.818161    2567 generic.go:356] "Generic (PLEG): container finished" podID="8c528839-a6fc-4085-b779-87826bb6588f" containerID="08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9" exitCode=0
Sep 18 20:25:30.818384 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.818205    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d" event={"ID":"8c528839-a6fc-4085-b779-87826bb6588f","Type":"ContainerDied","Data":"08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9"}
Sep 18 20:25:30.818384 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.818218    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"
Sep 18 20:25:30.818384 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.818255    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d" event={"ID":"8c528839-a6fc-4085-b779-87826bb6588f","Type":"ContainerDied","Data":"8fbddfa0659920c4467f8dbfcc141be1eecde69cb6ae92452c217d7506e069e8"}
Sep 18 20:25:30.818384 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.818279    2567 scope.go:117] "RemoveContainer" containerID="08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9"
Sep 18 20:25:30.828391 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.828362    2567 scope.go:117] "RemoveContainer" containerID="08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9"
Sep 18 20:25:30.828705 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:25:30.828662    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9\": container with ID starting with 08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9 not found: ID does not exist" containerID="08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9"
Sep 18 20:25:30.828800 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.828711    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9"} err="failed to get container status \"08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9\": rpc error: code = NotFound desc = could not find container \"08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9\": container with ID starting with 08af5057e50c26ab9d718f97116b828003ab4464835add5a89f395041b2e87d9 not found: ID does not exist"
Sep 18 20:25:30.920207 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.920175    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c528839-a6fc-4085-b779-87826bb6588f-plugins-config-volume\") pod \"8c528839-a6fc-4085-b779-87826bb6588f\" (UID: \"8c528839-a6fc-4085-b779-87826bb6588f\") "
Sep 18 20:25:30.920351 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.920215    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm5wk\" (UniqueName: \"kubernetes.io/projected/8c528839-a6fc-4085-b779-87826bb6588f-kube-api-access-pm5wk\") pod \"8c528839-a6fc-4085-b779-87826bb6588f\" (UID: \"8c528839-a6fc-4085-b779-87826bb6588f\") "
Sep 18 20:25:30.920555 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.920533    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c528839-a6fc-4085-b779-87826bb6588f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8c528839-a6fc-4085-b779-87826bb6588f" (UID: "8c528839-a6fc-4085-b779-87826bb6588f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:25:30.922333 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:30.922315    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c528839-a6fc-4085-b779-87826bb6588f-kube-api-access-pm5wk" (OuterVolumeSpecName: "kube-api-access-pm5wk") pod "8c528839-a6fc-4085-b779-87826bb6588f" (UID: "8c528839-a6fc-4085-b779-87826bb6588f"). InnerVolumeSpecName "kube-api-access-pm5wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:25:31.022213 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:31.022128    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c528839-a6fc-4085-b779-87826bb6588f-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:31.022213 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:31.022162    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm5wk\" (UniqueName: \"kubernetes.io/projected/8c528839-a6fc-4085-b779-87826bb6588f-kube-api-access-pm5wk\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:31.143164 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:31.143130    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"]
Sep 18 20:25:31.149531 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:31.149502    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-5319c6-6c4b795659-2bl2d"]
Sep 18 20:25:31.701149 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:31.701117    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c528839-a6fc-4085-b779-87826bb6588f" path="/var/lib/kubelet/pods/8c528839-a6fc-4085-b779-87826bb6588f/volumes"
Sep 18 20:25:48.449948 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.449904    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"]
Sep 18 20:25:48.450522 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.450132    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl" podUID="158d6542-f9cf-4d2a-b1bc-0623419a4895" containerName="maas-api" containerID="cri-o://4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870" gracePeriod=30
Sep 18 20:25:48.523485 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.523455    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"]
Sep 18 20:25:48.523870 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.523675    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4" podUID="df0ae129-30f2-42ec-8539-f9d15b880efa" containerName="payload-processing" containerID="cri-o://6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b" gracePeriod=30
Sep 18 20:25:48.775131 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.775103    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:25:48.801302 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.801274    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-maas-api-tls\") pod \"158d6542-f9cf-4d2a-b1bc-0623419a4895\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") "
Sep 18 20:25:48.801455 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.801301    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:25:48.801455 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.801324    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/158d6542-f9cf-4d2a-b1bc-0623419a4895-openshift-service-ca\") pod \"158d6542-f9cf-4d2a-b1bc-0623419a4895\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") "
Sep 18 20:25:48.801455 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.801357    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-metrics-certs\") pod \"158d6542-f9cf-4d2a-b1bc-0623419a4895\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") "
Sep 18 20:25:48.801455 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.801393    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mpb8\" (UniqueName: \"kubernetes.io/projected/158d6542-f9cf-4d2a-b1bc-0623419a4895-kube-api-access-8mpb8\") pod \"158d6542-f9cf-4d2a-b1bc-0623419a4895\" (UID: \"158d6542-f9cf-4d2a-b1bc-0623419a4895\") "
Sep 18 20:25:48.802070 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.802037    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/158d6542-f9cf-4d2a-b1bc-0623419a4895-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "158d6542-f9cf-4d2a-b1bc-0623419a4895" (UID: "158d6542-f9cf-4d2a-b1bc-0623419a4895"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:25:48.804297 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.804256    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "158d6542-f9cf-4d2a-b1bc-0623419a4895" (UID: "158d6542-f9cf-4d2a-b1bc-0623419a4895"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:25:48.804462 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.804416    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "158d6542-f9cf-4d2a-b1bc-0623419a4895" (UID: "158d6542-f9cf-4d2a-b1bc-0623419a4895"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:25:48.805004 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.804977    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/158d6542-f9cf-4d2a-b1bc-0623419a4895-kube-api-access-8mpb8" (OuterVolumeSpecName: "kube-api-access-8mpb8") pod "158d6542-f9cf-4d2a-b1bc-0623419a4895" (UID: "158d6542-f9cf-4d2a-b1bc-0623419a4895"). InnerVolumeSpecName "kube-api-access-8mpb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:25:48.902356 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.902323    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df0ae129-30f2-42ec-8539-f9d15b880efa-plugins-config-volume\") pod \"df0ae129-30f2-42ec-8539-f9d15b880efa\" (UID: \"df0ae129-30f2-42ec-8539-f9d15b880efa\") "
Sep 18 20:25:48.902531 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.902411    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5kfn\" (UniqueName: \"kubernetes.io/projected/df0ae129-30f2-42ec-8539-f9d15b880efa-kube-api-access-c5kfn\") pod \"df0ae129-30f2-42ec-8539-f9d15b880efa\" (UID: \"df0ae129-30f2-42ec-8539-f9d15b880efa\") "
Sep 18 20:25:48.902760 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.902738    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df0ae129-30f2-42ec-8539-f9d15b880efa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "df0ae129-30f2-42ec-8539-f9d15b880efa" (UID: "df0ae129-30f2-42ec-8539-f9d15b880efa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:25:48.902816 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.902773    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-maas-api-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:48.902816 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.902796    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/158d6542-f9cf-4d2a-b1bc-0623419a4895-openshift-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:48.902816 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.902810    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158d6542-f9cf-4d2a-b1bc-0623419a4895-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:48.902912 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.902824    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mpb8\" (UniqueName: \"kubernetes.io/projected/158d6542-f9cf-4d2a-b1bc-0623419a4895-kube-api-access-8mpb8\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:48.904433 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.904415    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df0ae129-30f2-42ec-8539-f9d15b880efa-kube-api-access-c5kfn" (OuterVolumeSpecName: "kube-api-access-c5kfn") pod "df0ae129-30f2-42ec-8539-f9d15b880efa" (UID: "df0ae129-30f2-42ec-8539-f9d15b880efa"). InnerVolumeSpecName "kube-api-access-c5kfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:25:48.917827 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.917801    2567 generic.go:356] "Generic (PLEG): container finished" podID="158d6542-f9cf-4d2a-b1bc-0623419a4895" containerID="4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870" exitCode=0
Sep 18 20:25:48.917969 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.917842    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"
Sep 18 20:25:48.917969 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.917856    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl" event={"ID":"158d6542-f9cf-4d2a-b1bc-0623419a4895","Type":"ContainerDied","Data":"4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870"}
Sep 18 20:25:48.917969 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.917907    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl" event={"ID":"158d6542-f9cf-4d2a-b1bc-0623419a4895","Type":"ContainerDied","Data":"bfb1fcd1f9a63080e7bb31d8b654e058f1bf3b7106b19fac9227e4de694a1018"}
Sep 18 20:25:48.917969 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.917928    2567 scope.go:117] "RemoveContainer" containerID="4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870"
Sep 18 20:25:48.919671 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.919642    2567 generic.go:356] "Generic (PLEG): container finished" podID="df0ae129-30f2-42ec-8539-f9d15b880efa" containerID="6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b" exitCode=0
Sep 18 20:25:48.919795 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.919710    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"
Sep 18 20:25:48.919795 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.919714    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4" event={"ID":"df0ae129-30f2-42ec-8539-f9d15b880efa","Type":"ContainerDied","Data":"6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b"}
Sep 18 20:25:48.920055 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.919819    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4" event={"ID":"df0ae129-30f2-42ec-8539-f9d15b880efa","Type":"ContainerDied","Data":"9d9cd40c3418d400c98c1be811382ec82ac2e86f8fd9b020f10547c1a9e6f137"}
Sep 18 20:25:48.930594 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.930541    2567 scope.go:117] "RemoveContainer" containerID="4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870"
Sep 18 20:25:48.931242 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:25:48.931215    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870\": container with ID starting with 4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870 not found: ID does not exist" containerID="4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870"
Sep 18 20:25:48.931344 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.931253    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870"} err="failed to get container status \"4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870\": rpc error: code = NotFound desc = could not find container \"4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870\": container with ID starting with 4a67d21aa94a7486291eafbffdece69699dce9f20ffdc88b97751f0d0397c870 not found: ID does not exist"
Sep 18 20:25:48.931344 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.931277    2567 scope.go:117] "RemoveContainer" containerID="6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b"
Sep 18 20:25:48.939900 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.939877    2567 scope.go:117] "RemoveContainer" containerID="6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b"
Sep 18 20:25:48.940197 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:25:48.940179    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b\": container with ID starting with 6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b not found: ID does not exist" containerID="6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b"
Sep 18 20:25:48.940270 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.940203    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b"} err="failed to get container status \"6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b\": rpc error: code = NotFound desc = could not find container \"6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b\": container with ID starting with 6a45a4a1b9d7382502495cbe515b3392a1ab1f2832684cf145e7fcf881fa6d2b not found: ID does not exist"
Sep 18 20:25:48.952485 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.952466    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"]
Sep 18 20:25:48.959611 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.959590    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c5206e-599989fc9d-tsqnl"]
Sep 18 20:25:48.981393 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.981330    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"]
Sep 18 20:25:48.988587 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:48.988566    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-c5206e-7c9b5b484f-f2xn4"]
Sep 18 20:25:49.011382 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:49.011360    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df0ae129-30f2-42ec-8539-f9d15b880efa-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:49.011382 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:49.011381    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5kfn\" (UniqueName: \"kubernetes.io/projected/df0ae129-30f2-42ec-8539-f9d15b880efa-kube-api-access-c5kfn\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:25:49.701597 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:49.701559    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="158d6542-f9cf-4d2a-b1bc-0623419a4895" path="/var/lib/kubelet/pods/158d6542-f9cf-4d2a-b1bc-0623419a4895/volumes"
Sep 18 20:25:49.702180 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:25:49.702159    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df0ae129-30f2-42ec-8539-f9d15b880efa" path="/var/lib/kubelet/pods/df0ae129-30f2-42ec-8539-f9d15b880efa/volumes"
Sep 18 20:26:17.919309 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919278    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"]
Sep 18 20:26:17.919761 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919746    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="158d6542-f9cf-4d2a-b1bc-0623419a4895" containerName="maas-api"
Sep 18 20:26:17.919761 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919762    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="158d6542-f9cf-4d2a-b1bc-0623419a4895" containerName="maas-api"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919774    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c528839-a6fc-4085-b779-87826bb6588f" containerName="payload-processing"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919780    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c528839-a6fc-4085-b779-87826bb6588f" containerName="payload-processing"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919786    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df0ae129-30f2-42ec-8539-f9d15b880efa" containerName="payload-processing"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919791    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="df0ae129-30f2-42ec-8539-f9d15b880efa" containerName="payload-processing"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919798    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e" containerName="payload-processing"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919803    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e" containerName="payload-processing"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919811    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" containerName="maas-api"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919817    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" containerName="maas-api"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919827    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29402ff9-6864-4cd2-8b7b-024dec9d07b4" containerName="main"
Sep 18 20:26:17.919859 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919832    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="29402ff9-6864-4cd2-8b7b-024dec9d07b4" containerName="main"
Sep 18 20:26:17.920164 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919913    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="158d6542-f9cf-4d2a-b1bc-0623419a4895" containerName="maas-api"
Sep 18 20:26:17.920164 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919921    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="df0ae129-30f2-42ec-8539-f9d15b880efa" containerName="payload-processing"
Sep 18 20:26:17.920164 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919929    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="29402ff9-6864-4cd2-8b7b-024dec9d07b4" containerName="main"
Sep 18 20:26:17.920164 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919935    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d8dedf-81c5-4ad4-ac1f-dac2c3a9c42e" containerName="payload-processing"
Sep 18 20:26:17.920164 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919945    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9c899ff-fd54-44f0-9c0c-ec9a71a221a9" containerName="maas-api"
Sep 18 20:26:17.920164 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.919953    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c528839-a6fc-4085-b779-87826bb6588f" containerName="payload-processing"
Sep 18 20:26:17.924034 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.924013    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:17.927420 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.927399    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-40a1a5\"/\"rate-test-model-40a1a5-kserve-self-signed-certs\""
Sep 18 20:26:17.927420 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.927411    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-40a1a5\"/\"default-dockercfg-5vgdp\""
Sep 18 20:26:17.927588 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.927405    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-40a1a5\"/\"kube-root-ca.crt\""
Sep 18 20:26:17.927588 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.927417    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-40a1a5\"/\"openshift-service-ca.crt\""
Sep 18 20:26:17.945507 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:17.945485    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"]
Sep 18 20:26:18.095781 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.095748    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-home\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.095939 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.095805    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tls-certs\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.095939 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.095850    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-model-cache\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.095939 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.095875    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64q5c\" (UniqueName: \"kubernetes.io/projected/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-kube-api-access-64q5c\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.096048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.095942    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tmp-dir\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.096048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.095977    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-dshm\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.197159 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.197086    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tmp-dir\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.197159 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.197121    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-dshm\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.197323 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.197176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-home\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.197323 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.197216    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tls-certs\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.197323 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.197254    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-model-cache\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.197323 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.197288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64q5c\" (UniqueName: \"kubernetes.io/projected/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-kube-api-access-64q5c\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.197556 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.197465    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tmp-dir\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.197556 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.197543    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-home\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.197726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.197680    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-model-cache\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.199455 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.199438    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-dshm\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.199747 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.199730    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tls-certs\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.206511 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.206492    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64q5c\" (UniqueName: \"kubernetes.io/projected/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-kube-api-access-64q5c\") pod \"rate-test-model-40a1a5-kserve-64c8c49cb7-lj569\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") " pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.233391 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.233371    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:18.510027 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:18.510000    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"]
Sep 18 20:26:18.511157 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:26:18.511131    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0df5a9d6_f59b_4ed4_a956_74ea66fd27fe.slice/crio-fc059d135bfa456218b4a91a89967d42fd9c9309e5199374ea884d8456c12187 WatchSource:0}: Error finding container fc059d135bfa456218b4a91a89967d42fd9c9309e5199374ea884d8456c12187: Status 404 returned error can't find the container with id fc059d135bfa456218b4a91a89967d42fd9c9309e5199374ea884d8456c12187
Sep 18 20:26:19.088182 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:19.088149    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569" event={"ID":"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe","Type":"ContainerStarted","Data":"2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af"}
Sep 18 20:26:19.088580 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:19.088193    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569" event={"ID":"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe","Type":"ContainerStarted","Data":"fc059d135bfa456218b4a91a89967d42fd9c9309e5199374ea884d8456c12187"}
Sep 18 20:26:19.115103 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:19.115058    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569" podStartSLOduration=2.115043442 podStartE2EDuration="2.115043442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:26:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:26:19.11277293 +0000 UTC m=+1759.673356389" watchObservedRunningTime="2026-09-18 20:26:19.115043442 +0000 UTC m=+1759.675626899"
Sep 18 20:26:28.234297 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:28.234255    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:28.234797 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:28.234337    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:28.246651 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:28.246628    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:26:29.150519 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:26:29.150493    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:27:01.795521 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:01.795491    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:27:01.802991 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:01.802968    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:27:26.979731 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:26.979681    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"]
Sep 18 20:27:35.872522 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:35.872485    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"]
Sep 18 20:27:35.876074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:35.876058    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:27:35.878782 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:35.878755    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-de7ea3\""
Sep 18 20:27:35.878919 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:35.878858    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-de7ea3-dockercfg-st6sw\""
Sep 18 20:27:35.886880 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:35.886857    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"]
Sep 18 20:27:35.958779 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:35.958732    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918f2e62-c804-49af-9e82-68b66e1944d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb\" (UID: \"918f2e62-c804-49af-9e82-68b66e1944d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:27:35.958942 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:35.958806    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr6g4\" (UniqueName: \"kubernetes.io/projected/918f2e62-c804-49af-9e82-68b66e1944d8-kube-api-access-gr6g4\") pod \"payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb\" (UID: \"918f2e62-c804-49af-9e82-68b66e1944d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:27:36.060340 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:36.060304    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918f2e62-c804-49af-9e82-68b66e1944d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb\" (UID: \"918f2e62-c804-49af-9e82-68b66e1944d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:27:36.060508 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:36.060367    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr6g4\" (UniqueName: \"kubernetes.io/projected/918f2e62-c804-49af-9e82-68b66e1944d8-kube-api-access-gr6g4\") pod \"payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb\" (UID: \"918f2e62-c804-49af-9e82-68b66e1944d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:27:36.061146 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:36.061127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918f2e62-c804-49af-9e82-68b66e1944d8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb\" (UID: \"918f2e62-c804-49af-9e82-68b66e1944d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:27:36.069399 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:36.069363    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr6g4\" (UniqueName: \"kubernetes.io/projected/918f2e62-c804-49af-9e82-68b66e1944d8-kube-api-access-gr6g4\") pod \"payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb\" (UID: \"918f2e62-c804-49af-9e82-68b66e1944d8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:27:36.186658 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:36.186571    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:27:36.460918 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:36.460890    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"]
Sep 18 20:27:36.463490 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:27:36.463462    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod918f2e62_c804_49af_9e82_68b66e1944d8.slice/crio-3d24dba2d9b451a8284e571dee1f4c5576d511c70be69736226ee5f87c88f9b4 WatchSource:0}: Error finding container 3d24dba2d9b451a8284e571dee1f4c5576d511c70be69736226ee5f87c88f9b4: Status 404 returned error can't find the container with id 3d24dba2d9b451a8284e571dee1f4c5576d511c70be69736226ee5f87c88f9b4
Sep 18 20:27:36.494034 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:36.494002    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb" event={"ID":"918f2e62-c804-49af-9e82-68b66e1944d8","Type":"ContainerStarted","Data":"3d24dba2d9b451a8284e571dee1f4c5576d511c70be69736226ee5f87c88f9b4"}
Sep 18 20:27:37.501393 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:37.501350    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb" event={"ID":"918f2e62-c804-49af-9e82-68b66e1944d8","Type":"ContainerStarted","Data":"3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b"}
Sep 18 20:27:37.501826 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:37.501409    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:27:37.521595 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:37.521547    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb" podStartSLOduration=2.521532175 podStartE2EDuration="2.521532175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:27:37.520041317 +0000 UTC m=+1838.080624774" watchObservedRunningTime="2026-09-18 20:27:37.521532175 +0000 UTC m=+1838.082115679"
Sep 18 20:27:41.998875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:41.998815    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569" podUID="0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" containerName="main" containerID="cri-o://2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af" gracePeriod=45
Sep 18 20:27:42.253449 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.253391    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:27:42.323492 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.323462    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tmp-dir\") pod \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") "
Sep 18 20:27:42.323633 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.323507    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-dshm\") pod \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") "
Sep 18 20:27:42.323633 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.323532    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-home\") pod \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") "
Sep 18 20:27:42.323752 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.323676    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tls-certs\") pod \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") "
Sep 18 20:27:42.323792 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.323768    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" (UID: "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:42.323792 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.323783    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64q5c\" (UniqueName: \"kubernetes.io/projected/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-kube-api-access-64q5c\") pod \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") "
Sep 18 20:27:42.323872 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.323808    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-home" (OuterVolumeSpecName: "home") pod "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" (UID: "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:42.323872 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.323822    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-model-cache\") pod \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\" (UID: \"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe\") "
Sep 18 20:27:42.324102 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.324077    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-model-cache" (OuterVolumeSpecName: "model-cache") pod "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" (UID: "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:42.324238 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.324217    2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tmp-dir\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:27:42.324309 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.324241    2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-home\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:27:42.324309 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.324250    2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-model-cache\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:27:42.325853 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.325828    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-dshm" (OuterVolumeSpecName: "dshm") pod "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" (UID: "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:42.325938 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.325902    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" (UID: "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:27:42.325938 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.325904    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-kube-api-access-64q5c" (OuterVolumeSpecName: "kube-api-access-64q5c") pod "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" (UID: "0df5a9d6-f59b-4ed4-a956-74ea66fd27fe"). InnerVolumeSpecName "kube-api-access-64q5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:42.425530 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.425502    2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-dshm\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:27:42.425530 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.425527    2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-tls-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:27:42.425672 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.425542    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64q5c\" (UniqueName: \"kubernetes.io/projected/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe-kube-api-access-64q5c\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:27:42.529342 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.529262    2567 generic.go:356] "Generic (PLEG): container finished" podID="0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" containerID="2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af" exitCode=0
Sep 18 20:27:42.529342 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.529315    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"
Sep 18 20:27:42.529493 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.529339    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569" event={"ID":"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe","Type":"ContainerDied","Data":"2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af"}
Sep 18 20:27:42.529493 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.529380    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569" event={"ID":"0df5a9d6-f59b-4ed4-a956-74ea66fd27fe","Type":"ContainerDied","Data":"fc059d135bfa456218b4a91a89967d42fd9c9309e5199374ea884d8456c12187"}
Sep 18 20:27:42.529493 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.529406    2567 scope.go:117] "RemoveContainer" containerID="2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af"
Sep 18 20:27:42.548557 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.548536    2567 scope.go:117] "RemoveContainer" containerID="2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af"
Sep 18 20:27:42.548843 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:27:42.548822    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af\": container with ID starting with 2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af not found: ID does not exist" containerID="2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af"
Sep 18 20:27:42.548902 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.548852    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af"} err="failed to get container status \"2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af\": rpc error: code = NotFound desc = could not find container \"2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af\": container with ID starting with 2998025906f35194adb715f3ca32224f7981a75744b86d7ad3e5054230d715af not found: ID does not exist"
Sep 18 20:27:42.559251 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.559226    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"]
Sep 18 20:27:42.562240 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:42.562215    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w4-40a1a5/rate-test-model-40a1a5-kserve-64c8c49cb7-lj569"]
Sep 18 20:27:43.699504 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:43.699470    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" path="/var/lib/kubelet/pods/0df5a9d6-f59b-4ed4-a956-74ea66fd27fe/volumes"
Sep 18 20:27:48.509672 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:27:48.509643    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:28:10.074809 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.074778    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"]
Sep 18 20:28:10.075257 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.075243    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" containerName="main"
Sep 18 20:28:10.075257 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.075258    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" containerName="main"
Sep 18 20:28:10.075419 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.075336    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0df5a9d6-f59b-4ed4-a956-74ea66fd27fe" containerName="main"
Sep 18 20:28:10.079765 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.079744    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.082835 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.082812    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-a1a0a2-serving-cert\""
Sep 18 20:28:10.102305 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.102275    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"]
Sep 18 20:28:10.105922 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.105896    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-metrics-certs\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.106061 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.105937    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb5b0ad-ff73-4162-9817-a201db4a139b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.106061 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.106015    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tgbz\" (UniqueName: \"kubernetes.io/projected/bcb5b0ad-ff73-4162-9817-a201db4a139b-kube-api-access-6tgbz\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.106185 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.106061    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.207074 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.207043    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.207248 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.207160    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-metrics-certs\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.207248 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.207194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb5b0ad-ff73-4162-9817-a201db4a139b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.207248 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:28:10.207203    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-serving-cert: secret "maas-api-e2e-ipp-b-a1a0a2-serving-cert" not found
Sep 18 20:28:10.207426 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.207247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tgbz\" (UniqueName: \"kubernetes.io/projected/bcb5b0ad-ff73-4162-9817-a201db4a139b-kube-api-access-6tgbz\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.207426 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:28:10.207258    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-maas-api-tls podName:bcb5b0ad-ff73-4162-9817-a201db4a139b nodeName:}" failed. No retries permitted until 2026-09-18 20:28:10.707243294 +0000 UTC m=+1871.267826742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-maas-api-tls") pod "maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt" (UID: "bcb5b0ad-ff73-4162-9817-a201db4a139b") : secret "maas-api-e2e-ipp-b-a1a0a2-serving-cert" not found
Sep 18 20:28:10.208115 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.208085    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb5b0ad-ff73-4162-9817-a201db4a139b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.210236 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.210215    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-metrics-certs\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.216875 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.216849    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tgbz\" (UniqueName: \"kubernetes.io/projected/bcb5b0ad-ff73-4162-9817-a201db4a139b-kube-api-access-6tgbz\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.712457 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.712424    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.714852 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.714829    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:10.990760 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:10.990653    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:11.058763 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.058732    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"]
Sep 18 20:28:11.064097 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.064078    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:11.066938 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.066907    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-a1a0a2-dockercfg-8z6c7\""
Sep 18 20:28:11.067048 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.066912    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-a1a0a2\""
Sep 18 20:28:11.076671 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.076590    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"]
Sep 18 20:28:11.116654 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.116624    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1794d226-c519-4076-ab2f-a751b598cf2a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc\" (UID: \"1794d226-c519-4076-ab2f-a751b598cf2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:11.116810 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.116673    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvcv2\" (UniqueName: \"kubernetes.io/projected/1794d226-c519-4076-ab2f-a751b598cf2a-kube-api-access-pvcv2\") pod \"payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc\" (UID: \"1794d226-c519-4076-ab2f-a751b598cf2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:11.217394 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.217362    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1794d226-c519-4076-ab2f-a751b598cf2a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc\" (UID: \"1794d226-c519-4076-ab2f-a751b598cf2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:11.217550 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.217411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvcv2\" (UniqueName: \"kubernetes.io/projected/1794d226-c519-4076-ab2f-a751b598cf2a-kube-api-access-pvcv2\") pod \"payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc\" (UID: \"1794d226-c519-4076-ab2f-a751b598cf2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:11.218350 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.218327    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1794d226-c519-4076-ab2f-a751b598cf2a-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc\" (UID: \"1794d226-c519-4076-ab2f-a751b598cf2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:11.226175 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.226144    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvcv2\" (UniqueName: \"kubernetes.io/projected/1794d226-c519-4076-ab2f-a751b598cf2a-kube-api-access-pvcv2\") pod \"payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc\" (UID: \"1794d226-c519-4076-ab2f-a751b598cf2a\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:11.264404 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.264375    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"]
Sep 18 20:28:11.266007 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:28:11.265984    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb5b0ad_ff73_4162_9817_a201db4a139b.slice/crio-c712ebafb57a1d9cbc2855e5f4a248cc837c24bf0b0e41a88243240de657490b WatchSource:0}: Error finding container c712ebafb57a1d9cbc2855e5f4a248cc837c24bf0b0e41a88243240de657490b: Status 404 returned error can't find the container with id c712ebafb57a1d9cbc2855e5f4a248cc837c24bf0b0e41a88243240de657490b
Sep 18 20:28:11.380524 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.380494    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:11.658392 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.658365    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"]
Sep 18 20:28:11.660586 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:28:11.660566    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1794d226_c519_4076_ab2f_a751b598cf2a.slice/crio-6e4d8a8e6c2def48e2566172cc12949b4507900527b6da8bf72a94aed7cf8589 WatchSource:0}: Error finding container 6e4d8a8e6c2def48e2566172cc12949b4507900527b6da8bf72a94aed7cf8589: Status 404 returned error can't find the container with id 6e4d8a8e6c2def48e2566172cc12949b4507900527b6da8bf72a94aed7cf8589
Sep 18 20:28:11.684925 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.684895    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt" event={"ID":"bcb5b0ad-ff73-4162-9817-a201db4a139b","Type":"ContainerStarted","Data":"c712ebafb57a1d9cbc2855e5f4a248cc837c24bf0b0e41a88243240de657490b"}
Sep 18 20:28:11.686138 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:11.686118    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc" event={"ID":"1794d226-c519-4076-ab2f-a751b598cf2a","Type":"ContainerStarted","Data":"6e4d8a8e6c2def48e2566172cc12949b4507900527b6da8bf72a94aed7cf8589"}
Sep 18 20:28:12.692515 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:12.692481    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc" event={"ID":"1794d226-c519-4076-ab2f-a751b598cf2a","Type":"ContainerStarted","Data":"825cf7dc710254690c40bb9974744d4a073662859057e5bdd22346deed3ec7f6"}
Sep 18 20:28:12.692987 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:12.692589    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:12.694190 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:12.694164    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt" event={"ID":"bcb5b0ad-ff73-4162-9817-a201db4a139b","Type":"ContainerStarted","Data":"9836bdef19d0bfc9257abacc41fdaa9b1392786f4176071890058f4934ea5fcd"}
Sep 18 20:28:12.694309 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:12.694293    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:12.729880 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:12.729838    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc" podStartSLOduration=1.729825279 podStartE2EDuration="1.729825279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:28:12.728922915 +0000 UTC m=+1873.289506373" watchObservedRunningTime="2026-09-18 20:28:12.729825279 +0000 UTC m=+1873.290408736"
Sep 18 20:28:12.783388 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:12.783320    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt" podStartSLOduration=2.389573214 podStartE2EDuration="2.783296478s" totalImagesPullingTime="393.723264ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:28:12.778132648 +0000 UTC m=+1873.338716104" watchObservedRunningTime="2026-09-18 20:28:12.783296478 +0000 UTC m=+1873.343879936"
Sep 18 20:28:18.705218 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:18.705186    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:28:23.701607 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:23.701576    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:28:42.007611 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:42.007576    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xr9rk_056b9973-eb31-492f-b1d2-b2e3877abf2d/payload-processing/0.log"
Sep 18 20:28:42.119722 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:28:42.119660    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc_1794d226-c519-4076-ab2f-a751b598cf2a/payload-processing/0.log"
Sep 18 20:29:18.298921 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:18.298888    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xr9rk_056b9973-eb31-492f-b1d2-b2e3877abf2d/payload-processing/0.log"
Sep 18 20:29:18.419968 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:18.419945    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc_1794d226-c519-4076-ab2f-a751b598cf2a/payload-processing/0.log"
Sep 18 20:29:18.536854 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:18.536830    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc_1794d226-c519-4076-ab2f-a751b598cf2a/payload-processing/0.log"
Sep 18 20:29:28.430250 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.430218    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"]
Sep 18 20:29:28.433847 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.433830    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:29:28.436334 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.436313    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-876382-dockercfg-s6f22\""
Sep 18 20:29:28.436463 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.436421    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-876382\""
Sep 18 20:29:28.447025 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.447001    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"]
Sep 18 20:29:28.559150 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.559117    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9\" (UID: \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:29:28.559150 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.559155    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47n4p\" (UniqueName: \"kubernetes.io/projected/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-kube-api-access-47n4p\") pod \"payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9\" (UID: \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:29:28.660395 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.660363    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9\" (UID: \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:29:28.660574 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.660402    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47n4p\" (UniqueName: \"kubernetes.io/projected/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-kube-api-access-47n4p\") pod \"payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9\" (UID: \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:29:28.661105 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.661085    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9\" (UID: \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:29:28.670916 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.670887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47n4p\" (UniqueName: \"kubernetes.io/projected/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-kube-api-access-47n4p\") pod \"payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9\" (UID: \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:29:28.748877 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:28.748788    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:29:29.022000 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:29.021969    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"]
Sep 18 20:29:29.024375 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:29:29.024344    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744d6a3f_6bc1_4299_a4e1_15ca43a93c1c.slice/crio-76fdee5bb230ac160005173e68bebd409ad8c9af2aa7d05923484eb0cd61213d WatchSource:0}: Error finding container 76fdee5bb230ac160005173e68bebd409ad8c9af2aa7d05923484eb0cd61213d: Status 404 returned error can't find the container with id 76fdee5bb230ac160005173e68bebd409ad8c9af2aa7d05923484eb0cd61213d
Sep 18 20:29:29.097871 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:29.097841    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9" event={"ID":"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c","Type":"ContainerStarted","Data":"aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe"}
Sep 18 20:29:29.097998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:29.097894    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9" event={"ID":"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c","Type":"ContainerStarted","Data":"76fdee5bb230ac160005173e68bebd409ad8c9af2aa7d05923484eb0cd61213d"}
Sep 18 20:29:29.097998 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:29.097919    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:29:29.117899 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:29.117851    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9" podStartSLOduration=1.117834796 podStartE2EDuration="1.117834796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:29:29.116442488 +0000 UTC m=+1949.677025946" watchObservedRunningTime="2026-09-18 20:29:29.117834796 +0000 UTC m=+1949.678418295"
Sep 18 20:29:40.106088 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:29:40.106053    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:30:00.289574 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:00.289488    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt"]
Sep 18 20:30:00.293664 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:00.293643    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt"
Sep 18 20:30:00.296124 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:00.296088    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bhssz\""
Sep 18 20:30:00.303952 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:00.303936    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt"
Sep 18 20:30:00.310302 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:00.310274    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt"]
Sep 18 20:30:00.598597 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:00.598571    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt"]
Sep 18 20:30:00.600247 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:30:00.600218    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f1577da_058b_45f2_91e2_710d551ad5cd.slice/crio-d1c28acdb7b7403a625e66df77884cc49711c92b9da727bdbeb8be9506ef6a69 WatchSource:0}: Error finding container d1c28acdb7b7403a625e66df77884cc49711c92b9da727bdbeb8be9506ef6a69: Status 404 returned error can't find the container with id d1c28acdb7b7403a625e66df77884cc49711c92b9da727bdbeb8be9506ef6a69
Sep 18 20:30:01.268424 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:01.268337    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt" event={"ID":"1f1577da-058b-45f2-91e2-710d551ad5cd","Type":"ContainerStarted","Data":"dcc22cb431bf0491f487a3298513c8064d6916e090dcbacacb8c7a5ede6bdae7"}
Sep 18 20:30:01.268424 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:01.268390    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt" event={"ID":"1f1577da-058b-45f2-91e2-710d551ad5cd","Type":"ContainerStarted","Data":"d1c28acdb7b7403a625e66df77884cc49711c92b9da727bdbeb8be9506ef6a69"}
Sep 18 20:30:01.287100 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:01.287055    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt" podStartSLOduration=1.287041458 podStartE2EDuration="1.287041458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:30:01.284538441 +0000 UTC m=+1981.845121899" watchObservedRunningTime="2026-09-18 20:30:01.287041458 +0000 UTC m=+1981.847624915"
Sep 18 20:30:02.274789 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:02.274755    2567 generic.go:356] "Generic (PLEG): container finished" podID="1f1577da-058b-45f2-91e2-710d551ad5cd" containerID="dcc22cb431bf0491f487a3298513c8064d6916e090dcbacacb8c7a5ede6bdae7" exitCode=0
Sep 18 20:30:02.275165 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:02.274836    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt" event={"ID":"1f1577da-058b-45f2-91e2-710d551ad5cd","Type":"ContainerDied","Data":"dcc22cb431bf0491f487a3298513c8064d6916e090dcbacacb8c7a5ede6bdae7"}
Sep 18 20:30:03.416433 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:03.416411    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt"
Sep 18 20:30:04.287221 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:04.287194    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt"
Sep 18 20:30:04.287221 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:04.287213    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt" event={"ID":"1f1577da-058b-45f2-91e2-710d551ad5cd","Type":"ContainerDied","Data":"d1c28acdb7b7403a625e66df77884cc49711c92b9da727bdbeb8be9506ef6a69"}
Sep 18 20:30:04.287420 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:04.287256    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1c28acdb7b7403a625e66df77884cc49711c92b9da727bdbeb8be9506ef6a69"
Sep 18 20:30:04.827660 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:04.827626    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"]
Sep 18 20:30:04.828125 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:04.827896    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9" podUID="744d6a3f-6bc1-4299-a4e1-15ca43a93c1c" containerName="payload-pre-processing" containerID="cri-o://aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe" gracePeriod=30
Sep 18 20:30:05.085429 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.085367    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:30:05.207526 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.207497    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-plugins-config-volume\") pod \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\" (UID: \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\") "
Sep 18 20:30:05.207743 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.207579    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47n4p\" (UniqueName: \"kubernetes.io/projected/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-kube-api-access-47n4p\") pod \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\" (UID: \"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c\") "
Sep 18 20:30:05.207893 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.207869    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "744d6a3f-6bc1-4299-a4e1-15ca43a93c1c" (UID: "744d6a3f-6bc1-4299-a4e1-15ca43a93c1c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:05.209628 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.209604    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-kube-api-access-47n4p" (OuterVolumeSpecName: "kube-api-access-47n4p") pod "744d6a3f-6bc1-4299-a4e1-15ca43a93c1c" (UID: "744d6a3f-6bc1-4299-a4e1-15ca43a93c1c"). InnerVolumeSpecName "kube-api-access-47n4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:05.293478 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.293444    2567 generic.go:356] "Generic (PLEG): container finished" podID="744d6a3f-6bc1-4299-a4e1-15ca43a93c1c" containerID="aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe" exitCode=0
Sep 18 20:30:05.293671 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.293489    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"
Sep 18 20:30:05.293671 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.293528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9" event={"ID":"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c","Type":"ContainerDied","Data":"aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe"}
Sep 18 20:30:05.293671 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.293585    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9" event={"ID":"744d6a3f-6bc1-4299-a4e1-15ca43a93c1c","Type":"ContainerDied","Data":"76fdee5bb230ac160005173e68bebd409ad8c9af2aa7d05923484eb0cd61213d"}
Sep 18 20:30:05.293671 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.293614    2567 scope.go:117] "RemoveContainer" containerID="aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe"
Sep 18 20:30:05.302953 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.302935    2567 scope.go:117] "RemoveContainer" containerID="aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe"
Sep 18 20:30:05.303214 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:30:05.303194    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe\": container with ID starting with aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe not found: ID does not exist" containerID="aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe"
Sep 18 20:30:05.303261 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.303223    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe"} err="failed to get container status \"aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe\": rpc error: code = NotFound desc = could not find container \"aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe\": container with ID starting with aac59f48ed37b03a453952adbd3f2693f21bfa8a4fe27f9deedb62df74b727fe not found: ID does not exist"
Sep 18 20:30:05.308491 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.308474    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:30:05.308549 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.308493    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47n4p\" (UniqueName: \"kubernetes.io/projected/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c-kube-api-access-47n4p\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:30:05.329806 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.329783    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"]
Sep 18 20:30:05.340315 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.340263    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-876382-7d8fc7cc66-6cnj9"]
Sep 18 20:30:05.699715 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:05.699652    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="744d6a3f-6bc1-4299-a4e1-15ca43a93c1c" path="/var/lib/kubelet/pods/744d6a3f-6bc1-4299-a4e1-15ca43a93c1c/volumes"
Sep 18 20:30:31.725485 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:31.725448    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"]
Sep 18 20:30:31.726134 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:31.725655    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb" podUID="918f2e62-c804-49af-9e82-68b66e1944d8" containerName="payload-pre-processing" containerID="cri-o://3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b" gracePeriod=30
Sep 18 20:30:31.981515 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:31.981451    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:30:32.165774 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.165736    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918f2e62-c804-49af-9e82-68b66e1944d8-plugins-config-volume\") pod \"918f2e62-c804-49af-9e82-68b66e1944d8\" (UID: \"918f2e62-c804-49af-9e82-68b66e1944d8\") "
Sep 18 20:30:32.165939 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.165828    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr6g4\" (UniqueName: \"kubernetes.io/projected/918f2e62-c804-49af-9e82-68b66e1944d8-kube-api-access-gr6g4\") pod \"918f2e62-c804-49af-9e82-68b66e1944d8\" (UID: \"918f2e62-c804-49af-9e82-68b66e1944d8\") "
Sep 18 20:30:32.166250 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.166223    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/918f2e62-c804-49af-9e82-68b66e1944d8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "918f2e62-c804-49af-9e82-68b66e1944d8" (UID: "918f2e62-c804-49af-9e82-68b66e1944d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:32.167990 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.167962    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/918f2e62-c804-49af-9e82-68b66e1944d8-kube-api-access-gr6g4" (OuterVolumeSpecName: "kube-api-access-gr6g4") pod "918f2e62-c804-49af-9e82-68b66e1944d8" (UID: "918f2e62-c804-49af-9e82-68b66e1944d8"). InnerVolumeSpecName "kube-api-access-gr6g4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:32.266953 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.266865    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/918f2e62-c804-49af-9e82-68b66e1944d8-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:30:32.266953 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.266890    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr6g4\" (UniqueName: \"kubernetes.io/projected/918f2e62-c804-49af-9e82-68b66e1944d8-kube-api-access-gr6g4\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:30:32.447886 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.447854    2567 generic.go:356] "Generic (PLEG): container finished" podID="918f2e62-c804-49af-9e82-68b66e1944d8" containerID="3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b" exitCode=0
Sep 18 20:30:32.448047 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.447896    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"
Sep 18 20:30:32.448047 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.447932    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb" event={"ID":"918f2e62-c804-49af-9e82-68b66e1944d8","Type":"ContainerDied","Data":"3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b"}
Sep 18 20:30:32.448047 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.448003    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb" event={"ID":"918f2e62-c804-49af-9e82-68b66e1944d8","Type":"ContainerDied","Data":"3d24dba2d9b451a8284e571dee1f4c5576d511c70be69736226ee5f87c88f9b4"}
Sep 18 20:30:32.448047 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.448024    2567 scope.go:117] "RemoveContainer" containerID="3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b"
Sep 18 20:30:32.460266 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.460183    2567 scope.go:117] "RemoveContainer" containerID="3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b"
Sep 18 20:30:32.461540 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:30:32.461513    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b\": container with ID starting with 3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b not found: ID does not exist" containerID="3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b"
Sep 18 20:30:32.461654 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.461546    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b"} err="failed to get container status \"3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b\": rpc error: code = NotFound desc = could not find container \"3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b\": container with ID starting with 3622f32507317a26a274ccc8ebeae9e9dfdc0a5e813042647801f3531a8bf65b not found: ID does not exist"
Sep 18 20:30:32.472800 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.472775    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"]
Sep 18 20:30:32.477985 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:32.477965    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-de7ea3-7b9c9945bb-8v2jb"]
Sep 18 20:30:33.700139 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:30:33.700108    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="918f2e62-c804-49af-9e82-68b66e1944d8" path="/var/lib/kubelet/pods/918f2e62-c804-49af-9e82-68b66e1944d8/volumes"
Sep 18 20:31:32.580385 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.580351    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"]
Sep 18 20:31:32.581041 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.580578    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt" podUID="bcb5b0ad-ff73-4162-9817-a201db4a139b" containerName="maas-api" containerID="cri-o://9836bdef19d0bfc9257abacc41fdaa9b1392786f4176071890058f4934ea5fcd" gracePeriod=30
Sep 18 20:31:32.658532 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.658498    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"]
Sep 18 20:31:32.658785 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.658753    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc" podUID="1794d226-c519-4076-ab2f-a751b598cf2a" containerName="payload-processing" containerID="cri-o://825cf7dc710254690c40bb9974744d4a073662859057e5bdd22346deed3ec7f6" gracePeriod=30
Sep 18 20:31:32.779240 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.779214    2567 generic.go:356] "Generic (PLEG): container finished" podID="1794d226-c519-4076-ab2f-a751b598cf2a" containerID="825cf7dc710254690c40bb9974744d4a073662859057e5bdd22346deed3ec7f6" exitCode=0
Sep 18 20:31:32.779347 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.779303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc" event={"ID":"1794d226-c519-4076-ab2f-a751b598cf2a","Type":"ContainerDied","Data":"825cf7dc710254690c40bb9974744d4a073662859057e5bdd22346deed3ec7f6"}
Sep 18 20:31:32.781581 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.781559    2567 generic.go:356] "Generic (PLEG): container finished" podID="bcb5b0ad-ff73-4162-9817-a201db4a139b" containerID="9836bdef19d0bfc9257abacc41fdaa9b1392786f4176071890058f4934ea5fcd" exitCode=0
Sep 18 20:31:32.781711 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.781636    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt" event={"ID":"bcb5b0ad-ff73-4162-9817-a201db4a139b","Type":"ContainerDied","Data":"9836bdef19d0bfc9257abacc41fdaa9b1392786f4176071890058f4934ea5fcd"}
Sep 18 20:31:32.906012 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.905984    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:31:32.950401 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:32.950376    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:31:33.061544 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.061509    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-maas-api-tls\") pod \"bcb5b0ad-ff73-4162-9817-a201db4a139b\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") "
Sep 18 20:31:33.061762 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.061581    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvcv2\" (UniqueName: \"kubernetes.io/projected/1794d226-c519-4076-ab2f-a751b598cf2a-kube-api-access-pvcv2\") pod \"1794d226-c519-4076-ab2f-a751b598cf2a\" (UID: \"1794d226-c519-4076-ab2f-a751b598cf2a\") "
Sep 18 20:31:33.061762 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.061627    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-metrics-certs\") pod \"bcb5b0ad-ff73-4162-9817-a201db4a139b\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") "
Sep 18 20:31:33.061918 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.061765    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb5b0ad-ff73-4162-9817-a201db4a139b-openshift-service-ca\") pod \"bcb5b0ad-ff73-4162-9817-a201db4a139b\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") "
Sep 18 20:31:33.061918 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.061810    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tgbz\" (UniqueName: \"kubernetes.io/projected/bcb5b0ad-ff73-4162-9817-a201db4a139b-kube-api-access-6tgbz\") pod \"bcb5b0ad-ff73-4162-9817-a201db4a139b\" (UID: \"bcb5b0ad-ff73-4162-9817-a201db4a139b\") "
Sep 18 20:31:33.061918 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.061882    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1794d226-c519-4076-ab2f-a751b598cf2a-plugins-config-volume\") pod \"1794d226-c519-4076-ab2f-a751b598cf2a\" (UID: \"1794d226-c519-4076-ab2f-a751b598cf2a\") "
Sep 18 20:31:33.062324 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.062284    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb5b0ad-ff73-4162-9817-a201db4a139b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bcb5b0ad-ff73-4162-9817-a201db4a139b" (UID: "bcb5b0ad-ff73-4162-9817-a201db4a139b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:31:33.062428 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.062362    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1794d226-c519-4076-ab2f-a751b598cf2a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1794d226-c519-4076-ab2f-a751b598cf2a" (UID: "1794d226-c519-4076-ab2f-a751b598cf2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:31:33.063999 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.063970    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb5b0ad-ff73-4162-9817-a201db4a139b-kube-api-access-6tgbz" (OuterVolumeSpecName: "kube-api-access-6tgbz") pod "bcb5b0ad-ff73-4162-9817-a201db4a139b" (UID: "bcb5b0ad-ff73-4162-9817-a201db4a139b"). InnerVolumeSpecName "kube-api-access-6tgbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:31:33.064172 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.064154    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bcb5b0ad-ff73-4162-9817-a201db4a139b" (UID: "bcb5b0ad-ff73-4162-9817-a201db4a139b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:31:33.064396 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.064382    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bcb5b0ad-ff73-4162-9817-a201db4a139b" (UID: "bcb5b0ad-ff73-4162-9817-a201db4a139b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:31:33.064501 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.064480    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1794d226-c519-4076-ab2f-a751b598cf2a-kube-api-access-pvcv2" (OuterVolumeSpecName: "kube-api-access-pvcv2") pod "1794d226-c519-4076-ab2f-a751b598cf2a" (UID: "1794d226-c519-4076-ab2f-a751b598cf2a"). InnerVolumeSpecName "kube-api-access-pvcv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:31:33.163613 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.163539    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb5b0ad-ff73-4162-9817-a201db4a139b-openshift-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:31:33.163613 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.163565    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tgbz\" (UniqueName: \"kubernetes.io/projected/bcb5b0ad-ff73-4162-9817-a201db4a139b-kube-api-access-6tgbz\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:31:33.163613 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.163577    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1794d226-c519-4076-ab2f-a751b598cf2a-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:31:33.163613 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.163586    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-maas-api-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:31:33.163613 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.163595    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvcv2\" (UniqueName: \"kubernetes.io/projected/1794d226-c519-4076-ab2f-a751b598cf2a-kube-api-access-pvcv2\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:31:33.163613 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.163605    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcb5b0ad-ff73-4162-9817-a201db4a139b-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:31:33.788397 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.788371    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"
Sep 18 20:31:33.788825 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.788407    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt" event={"ID":"bcb5b0ad-ff73-4162-9817-a201db4a139b","Type":"ContainerDied","Data":"c712ebafb57a1d9cbc2855e5f4a248cc837c24bf0b0e41a88243240de657490b"}
Sep 18 20:31:33.788825 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.788466    2567 scope.go:117] "RemoveContainer" containerID="9836bdef19d0bfc9257abacc41fdaa9b1392786f4176071890058f4934ea5fcd"
Sep 18 20:31:33.790021 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.789998    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc" event={"ID":"1794d226-c519-4076-ab2f-a751b598cf2a","Type":"ContainerDied","Data":"6e4d8a8e6c2def48e2566172cc12949b4507900527b6da8bf72a94aed7cf8589"}
Sep 18 20:31:33.790123 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.790040    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"
Sep 18 20:31:33.797549 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.797531    2567 scope.go:117] "RemoveContainer" containerID="825cf7dc710254690c40bb9974744d4a073662859057e5bdd22346deed3ec7f6"
Sep 18 20:31:33.826679 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.826658    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"]
Sep 18 20:31:33.835053 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.835026    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-a1a0a2-797c5469b7-j94zc"]
Sep 18 20:31:33.859051 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.859032    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"]
Sep 18 20:31:33.862729 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:33.862709    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a1a0a2-6749d5dcf-xm9kt"]
Sep 18 20:31:35.699086 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:35.699052    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1794d226-c519-4076-ab2f-a751b598cf2a" path="/var/lib/kubelet/pods/1794d226-c519-4076-ab2f-a751b598cf2a/volumes"
Sep 18 20:31:35.699524 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:35.699509    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcb5b0ad-ff73-4162-9817-a201db4a139b" path="/var/lib/kubelet/pods/bcb5b0ad-ff73-4162-9817-a201db4a139b/volumes"
Sep 18 20:31:51.754499 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:51.754465    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt"]
Sep 18 20:31:51.757779 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:51.757752    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w4-40a1a5-29829390-7d7nt"]
Sep 18 20:31:53.699755 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:31:53.699721    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f1577da-058b-45f2-91e2-710d551ad5cd" path="/var/lib/kubelet/pods/1f1577da-058b-45f2-91e2-710d551ad5cd/volumes"
Sep 18 20:32:01.832483 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:01.832455    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:32:01.841006 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:01.840986    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:32:10.633218 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.633184    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"]
Sep 18 20:32:10.633813 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.633393    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps" podUID="3536e404-87d3-439d-8b39-2469dcf63406" containerName="maas-api" containerID="cri-o://5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b" gracePeriod=30
Sep 18 20:32:10.717402 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.717366    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"]
Sep 18 20:32:10.717811 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.717621    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz" podUID="9447be53-157e-44da-a486-08b92a76cf33" containerName="payload-pre-processing" containerID="cri-o://e7111723d8ad99e14f57880a425678f4c7c259617697df139568497bb9a9d2ca" gracePeriod=30
Sep 18 20:32:10.968602 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.968578    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:32:10.990392 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.990297    2567 generic.go:356] "Generic (PLEG): container finished" podID="9447be53-157e-44da-a486-08b92a76cf33" containerID="e7111723d8ad99e14f57880a425678f4c7c259617697df139568497bb9a9d2ca" exitCode=0
Sep 18 20:32:10.990392 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.990340    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz" event={"ID":"9447be53-157e-44da-a486-08b92a76cf33","Type":"ContainerDied","Data":"e7111723d8ad99e14f57880a425678f4c7c259617697df139568497bb9a9d2ca"}
Sep 18 20:32:10.990721 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.990643    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz" event={"ID":"9447be53-157e-44da-a486-08b92a76cf33","Type":"ContainerDied","Data":"f09d059e773d9d592eb126978039d94391113f343a37da0cadc8bc0425064b61"}
Sep 18 20:32:10.990721 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.990702    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f09d059e773d9d592eb126978039d94391113f343a37da0cadc8bc0425064b61"
Sep 18 20:32:10.992301 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.992275    2567 generic.go:356] "Generic (PLEG): container finished" podID="3536e404-87d3-439d-8b39-2469dcf63406" containerID="5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b" exitCode=0
Sep 18 20:32:10.992419 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.992356    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"
Sep 18 20:32:10.992419 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.992378    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps" event={"ID":"3536e404-87d3-439d-8b39-2469dcf63406","Type":"ContainerDied","Data":"5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b"}
Sep 18 20:32:10.992532 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.992429    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps" event={"ID":"3536e404-87d3-439d-8b39-2469dcf63406","Type":"ContainerDied","Data":"3316363d461f2ab2654a327886db7ccb6882bfe16bc7effec29b24d8f1acc0a9"}
Sep 18 20:32:10.992532 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.992455    2567 scope.go:117] "RemoveContainer" containerID="5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b"
Sep 18 20:32:10.995796 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:10.995774    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:32:11.001933 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.001915    2567 scope.go:117] "RemoveContainer" containerID="5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b"
Sep 18 20:32:11.002178 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:32:11.002160    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b\": container with ID starting with 5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b not found: ID does not exist" containerID="5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b"
Sep 18 20:32:11.002236 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.002186    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b"} err="failed to get container status \"5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b\": rpc error: code = NotFound desc = could not find container \"5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b\": container with ID starting with 5396b53849030f3eaa250c47fd80a3210927c371eaf931236973c4c84c372c6b not found: ID does not exist"
Sep 18 20:32:11.022375 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.022351    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-maas-api-tls\") pod \"3536e404-87d3-439d-8b39-2469dcf63406\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") "
Sep 18 20:32:11.022470 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.022412    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3536e404-87d3-439d-8b39-2469dcf63406-openshift-service-ca\") pod \"3536e404-87d3-439d-8b39-2469dcf63406\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") "
Sep 18 20:32:11.022556 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.022541    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-metrics-certs\") pod \"3536e404-87d3-439d-8b39-2469dcf63406\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") "
Sep 18 20:32:11.022595 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.022572    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp4zt\" (UniqueName: \"kubernetes.io/projected/3536e404-87d3-439d-8b39-2469dcf63406-kube-api-access-xp4zt\") pod \"3536e404-87d3-439d-8b39-2469dcf63406\" (UID: \"3536e404-87d3-439d-8b39-2469dcf63406\") "
Sep 18 20:32:11.022784 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.022760    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3536e404-87d3-439d-8b39-2469dcf63406-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3536e404-87d3-439d-8b39-2469dcf63406" (UID: "3536e404-87d3-439d-8b39-2469dcf63406"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:11.022975 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.022958    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3536e404-87d3-439d-8b39-2469dcf63406-openshift-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:32:11.024417 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.024396    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3536e404-87d3-439d-8b39-2469dcf63406" (UID: "3536e404-87d3-439d-8b39-2469dcf63406"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:11.024543 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.024523    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3536e404-87d3-439d-8b39-2469dcf63406-kube-api-access-xp4zt" (OuterVolumeSpecName: "kube-api-access-xp4zt") pod "3536e404-87d3-439d-8b39-2469dcf63406" (UID: "3536e404-87d3-439d-8b39-2469dcf63406"). InnerVolumeSpecName "kube-api-access-xp4zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:32:11.024947 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.024928    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3536e404-87d3-439d-8b39-2469dcf63406" (UID: "3536e404-87d3-439d-8b39-2469dcf63406"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:11.123773 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.123742    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9447be53-157e-44da-a486-08b92a76cf33-plugins-config-volume\") pod \"9447be53-157e-44da-a486-08b92a76cf33\" (UID: \"9447be53-157e-44da-a486-08b92a76cf33\") "
Sep 18 20:32:11.123931 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.123842    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mchc\" (UniqueName: \"kubernetes.io/projected/9447be53-157e-44da-a486-08b92a76cf33-kube-api-access-4mchc\") pod \"9447be53-157e-44da-a486-08b92a76cf33\" (UID: \"9447be53-157e-44da-a486-08b92a76cf33\") "
Sep 18 20:32:11.124096 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.124074    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9447be53-157e-44da-a486-08b92a76cf33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9447be53-157e-44da-a486-08b92a76cf33" (UID: "9447be53-157e-44da-a486-08b92a76cf33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:32:11.124193 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.124175    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:32:11.124193 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.124191    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp4zt\" (UniqueName: \"kubernetes.io/projected/3536e404-87d3-439d-8b39-2469dcf63406-kube-api-access-xp4zt\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:32:11.124314 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.124201    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9447be53-157e-44da-a486-08b92a76cf33-plugins-config-volume\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:32:11.124314 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.124210    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3536e404-87d3-439d-8b39-2469dcf63406-maas-api-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:32:11.125765 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.125742    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9447be53-157e-44da-a486-08b92a76cf33-kube-api-access-4mchc" (OuterVolumeSpecName: "kube-api-access-4mchc") pod "9447be53-157e-44da-a486-08b92a76cf33" (UID: "9447be53-157e-44da-a486-08b92a76cf33"). InnerVolumeSpecName "kube-api-access-4mchc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:32:11.225480 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.225401    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mchc\" (UniqueName: \"kubernetes.io/projected/9447be53-157e-44da-a486-08b92a76cf33-kube-api-access-4mchc\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:32:11.326121 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.326092    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"]
Sep 18 20:32:11.332288 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.332259    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-81f19d-7f75df48c6-wvxps"]
Sep 18 20:32:11.705774 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.705740    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3536e404-87d3-439d-8b39-2469dcf63406" path="/var/lib/kubelet/pods/3536e404-87d3-439d-8b39-2469dcf63406/volumes"
Sep 18 20:32:11.998200 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:11.998131    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"
Sep 18 20:32:12.026620 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:12.026585    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"]
Sep 18 20:32:12.032695 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:12.032655    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-81f19d-654546cbd6-lh8sz"]
Sep 18 20:32:13.699595 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:13.699561    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9447be53-157e-44da-a486-08b92a76cf33" path="/var/lib/kubelet/pods/9447be53-157e-44da-a486-08b92a76cf33/volumes"
Sep 18 20:32:21.908326 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:21.908283    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-988c79b4c-kgxbn"]
Sep 18 20:32:21.908826 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:21.908561    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-988c79b4c-kgxbn" podUID="b638bdf6-e8fc-4111-980a-0acdeb104885" containerName="manager" containerID="cri-o://59fcd9199097a5f57e3007e7fc6e9a8a317f0335e776f7156aa8f4cc43c31ab6" gracePeriod=30
Sep 18 20:32:22.054132 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.054098    2567 generic.go:356] "Generic (PLEG): container finished" podID="b638bdf6-e8fc-4111-980a-0acdeb104885" containerID="59fcd9199097a5f57e3007e7fc6e9a8a317f0335e776f7156aa8f4cc43c31ab6" exitCode=0
Sep 18 20:32:22.054313 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.054151    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-988c79b4c-kgxbn" event={"ID":"b638bdf6-e8fc-4111-980a-0acdeb104885","Type":"ContainerDied","Data":"59fcd9199097a5f57e3007e7fc6e9a8a317f0335e776f7156aa8f4cc43c31ab6"}
Sep 18 20:32:22.164142 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.164090    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:32:22.328062 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.328022    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-metrics-certs\") pod \"b638bdf6-e8fc-4111-980a-0acdeb104885\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") "
Sep 18 20:32:22.328230 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.328121    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gndwk\" (UniqueName: \"kubernetes.io/projected/b638bdf6-e8fc-4111-980a-0acdeb104885-kube-api-access-gndwk\") pod \"b638bdf6-e8fc-4111-980a-0acdeb104885\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") "
Sep 18 20:32:22.328230 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.328147    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-webhook-cert\") pod \"b638bdf6-e8fc-4111-980a-0acdeb104885\" (UID: \"b638bdf6-e8fc-4111-980a-0acdeb104885\") "
Sep 18 20:32:22.330153 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.330123    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b638bdf6-e8fc-4111-980a-0acdeb104885" (UID: "b638bdf6-e8fc-4111-980a-0acdeb104885"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:22.330276 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.330155    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b638bdf6-e8fc-4111-980a-0acdeb104885" (UID: "b638bdf6-e8fc-4111-980a-0acdeb104885"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:22.330276 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.330166    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b638bdf6-e8fc-4111-980a-0acdeb104885-kube-api-access-gndwk" (OuterVolumeSpecName: "kube-api-access-gndwk") pod "b638bdf6-e8fc-4111-980a-0acdeb104885" (UID: "b638bdf6-e8fc-4111-980a-0acdeb104885"). InnerVolumeSpecName "kube-api-access-gndwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:32:22.429376 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.429286    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gndwk\" (UniqueName: \"kubernetes.io/projected/b638bdf6-e8fc-4111-980a-0acdeb104885-kube-api-access-gndwk\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:32:22.429376 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.429325    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-webhook-cert\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:32:22.429376 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:22.429336    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b638bdf6-e8fc-4111-980a-0acdeb104885-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:32:23.060973 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:23.060940    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-988c79b4c-kgxbn"
Sep 18 20:32:23.060973 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:23.060954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-988c79b4c-kgxbn" event={"ID":"b638bdf6-e8fc-4111-980a-0acdeb104885","Type":"ContainerDied","Data":"ede85fec258c35369dd0ca7d15185239b73da90f7c8b1a60c8d4efca928e0f96"}
Sep 18 20:32:23.061508 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:23.061007    2567 scope.go:117] "RemoveContainer" containerID="59fcd9199097a5f57e3007e7fc6e9a8a317f0335e776f7156aa8f4cc43c31ab6"
Sep 18 20:32:23.100033 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:23.100005    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-988c79b4c-kgxbn"]
Sep 18 20:32:23.102676 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:23.102655    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-988c79b4c-kgxbn"]
Sep 18 20:32:23.699410 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:32:23.699372    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b638bdf6-e8fc-4111-980a-0acdeb104885" path="/var/lib/kubelet/pods/b638bdf6-e8fc-4111-980a-0acdeb104885/volumes"
Sep 18 20:33:02.044472 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:02.044368    2567 scope.go:117] "RemoveContainer" containerID="e7111723d8ad99e14f57880a425678f4c7c259617697df139568497bb9a9d2ca"
Sep 18 20:33:23.986232 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986198    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-988c79b4c-wcwnq"]
Sep 18 20:33:23.986803 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986783    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f1577da-058b-45f2-91e2-710d551ad5cd" containerName="cleanup"
Sep 18 20:33:23.986904 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986804    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f1577da-058b-45f2-91e2-710d551ad5cd" containerName="cleanup"
Sep 18 20:33:23.986904 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986824    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb5b0ad-ff73-4162-9817-a201db4a139b" containerName="maas-api"
Sep 18 20:33:23.986904 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986832    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb5b0ad-ff73-4162-9817-a201db4a139b" containerName="maas-api"
Sep 18 20:33:23.986904 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986841    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1794d226-c519-4076-ab2f-a751b598cf2a" containerName="payload-processing"
Sep 18 20:33:23.986904 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986850    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1794d226-c519-4076-ab2f-a751b598cf2a" containerName="payload-processing"
Sep 18 20:33:23.986904 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986888    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b638bdf6-e8fc-4111-980a-0acdeb104885" containerName="manager"
Sep 18 20:33:23.986904 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986896    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b638bdf6-e8fc-4111-980a-0acdeb104885" containerName="manager"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986906    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9447be53-157e-44da-a486-08b92a76cf33" containerName="payload-pre-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986915    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9447be53-157e-44da-a486-08b92a76cf33" containerName="payload-pre-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986928    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="744d6a3f-6bc1-4299-a4e1-15ca43a93c1c" containerName="payload-pre-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986937    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="744d6a3f-6bc1-4299-a4e1-15ca43a93c1c" containerName="payload-pre-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986949    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3536e404-87d3-439d-8b39-2469dcf63406" containerName="maas-api"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986957    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="3536e404-87d3-439d-8b39-2469dcf63406" containerName="maas-api"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986966    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="918f2e62-c804-49af-9e82-68b66e1944d8" containerName="payload-pre-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.986975    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="918f2e62-c804-49af-9e82-68b66e1944d8" containerName="payload-pre-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.987063    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb5b0ad-ff73-4162-9817-a201db4a139b" containerName="maas-api"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.987074    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9447be53-157e-44da-a486-08b92a76cf33" containerName="payload-pre-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.987086    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b638bdf6-e8fc-4111-980a-0acdeb104885" containerName="manager"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.987097    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="744d6a3f-6bc1-4299-a4e1-15ca43a93c1c" containerName="payload-pre-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.987108    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1794d226-c519-4076-ab2f-a751b598cf2a" containerName="payload-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.987117    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="918f2e62-c804-49af-9e82-68b66e1944d8" containerName="payload-pre-processing"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.987126    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f1577da-058b-45f2-91e2-710d551ad5cd" containerName="cleanup"
Sep 18 20:33:23.987265 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.987133    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="3536e404-87d3-439d-8b39-2469dcf63406" containerName="maas-api"
Sep 18 20:33:23.992148 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:23.992126    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.000719 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.000698    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 18 20:33:24.001594 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.001579    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 18 20:33:24.001647 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.001601    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-j9h7q\""
Sep 18 20:33:24.001647 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.001639    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 18 20:33:24.016187 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.016166    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-988c79b4c-wcwnq"]
Sep 18 20:33:24.099481 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.099449    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-webhook-cert\") pod \"maas-controller-988c79b4c-wcwnq\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") " pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.099640 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.099486    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjkx\" (UniqueName: \"kubernetes.io/projected/36fa450d-79ef-416c-b192-c7c43c9d1a15-kube-api-access-hdjkx\") pod \"maas-controller-988c79b4c-wcwnq\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") " pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.099640 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.099553    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-metrics-certs\") pod \"maas-controller-988c79b4c-wcwnq\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") " pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.200594 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.200560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-webhook-cert\") pod \"maas-controller-988c79b4c-wcwnq\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") " pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.200594 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.200599    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjkx\" (UniqueName: \"kubernetes.io/projected/36fa450d-79ef-416c-b192-c7c43c9d1a15-kube-api-access-hdjkx\") pod \"maas-controller-988c79b4c-wcwnq\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") " pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.200811 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.200637    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-metrics-certs\") pod \"maas-controller-988c79b4c-wcwnq\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") " pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.203161 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.203131    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-webhook-cert\") pod \"maas-controller-988c79b4c-wcwnq\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") " pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.203304 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.203187    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-metrics-certs\") pod \"maas-controller-988c79b4c-wcwnq\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") " pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.210788 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.210763    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjkx\" (UniqueName: \"kubernetes.io/projected/36fa450d-79ef-416c-b192-c7c43c9d1a15-kube-api-access-hdjkx\") pod \"maas-controller-988c79b4c-wcwnq\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") " pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.301622 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.301540    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:24.580232 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.580202    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-988c79b4c-wcwnq"]
Sep 18 20:33:24.585098 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:33:24.585072    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36fa450d_79ef_416c_b192_c7c43c9d1a15.slice/crio-22d125e8e7a24ab1dcd801fc14f4f11f401b83414fe6bbcb0add14376a5e45fc WatchSource:0}: Error finding container 22d125e8e7a24ab1dcd801fc14f4f11f401b83414fe6bbcb0add14376a5e45fc: Status 404 returned error can't find the container with id 22d125e8e7a24ab1dcd801fc14f4f11f401b83414fe6bbcb0add14376a5e45fc
Sep 18 20:33:24.586509 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:24.586487    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:33:25.377348 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:25.377312    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-988c79b4c-wcwnq" event={"ID":"36fa450d-79ef-416c-b192-c7c43c9d1a15","Type":"ContainerStarted","Data":"2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc"}
Sep 18 20:33:25.377726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:25.377366    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-988c79b4c-wcwnq" event={"ID":"36fa450d-79ef-416c-b192-c7c43c9d1a15","Type":"ContainerStarted","Data":"22d125e8e7a24ab1dcd801fc14f4f11f401b83414fe6bbcb0add14376a5e45fc"}
Sep 18 20:33:25.377726 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:25.377460    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:25.399148 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:25.399101    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-988c79b4c-wcwnq" podStartSLOduration=1.792338956 podStartE2EDuration="2.399085097s" totalImagesPullingTime="606.746141ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:33:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:33:25.396566468 +0000 UTC m=+2185.957149925" watchObservedRunningTime="2026-09-18 20:33:25.399085097 +0000 UTC m=+2185.959668553"
Sep 18 20:33:36.384451 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:36.384422    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:45.702289 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:45.702254    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-988c79b4c-wcwnq"]
Sep 18 20:33:45.702722 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:45.702439    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-988c79b4c-wcwnq" podUID="36fa450d-79ef-416c-b192-c7c43c9d1a15" containerName="manager" containerID="cri-o://2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc" gracePeriod=30
Sep 18 20:33:45.952522 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:45.952461    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:46.009741 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.009709    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdjkx\" (UniqueName: \"kubernetes.io/projected/36fa450d-79ef-416c-b192-c7c43c9d1a15-kube-api-access-hdjkx\") pod \"36fa450d-79ef-416c-b192-c7c43c9d1a15\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") "
Sep 18 20:33:46.009923 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.009769    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-metrics-certs\") pod \"36fa450d-79ef-416c-b192-c7c43c9d1a15\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") "
Sep 18 20:33:46.009923 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.009809    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-webhook-cert\") pod \"36fa450d-79ef-416c-b192-c7c43c9d1a15\" (UID: \"36fa450d-79ef-416c-b192-c7c43c9d1a15\") "
Sep 18 20:33:46.011963 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.011934    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "36fa450d-79ef-416c-b192-c7c43c9d1a15" (UID: "36fa450d-79ef-416c-b192-c7c43c9d1a15"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:33:46.012067 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.011973    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "36fa450d-79ef-416c-b192-c7c43c9d1a15" (UID: "36fa450d-79ef-416c-b192-c7c43c9d1a15"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:33:46.012224 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.012196    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36fa450d-79ef-416c-b192-c7c43c9d1a15-kube-api-access-hdjkx" (OuterVolumeSpecName: "kube-api-access-hdjkx") pod "36fa450d-79ef-416c-b192-c7c43c9d1a15" (UID: "36fa450d-79ef-416c-b192-c7c43c9d1a15"). InnerVolumeSpecName "kube-api-access-hdjkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:33:46.111222 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.111192    2567 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-webhook-cert\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:33:46.111222 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.111220    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdjkx\" (UniqueName: \"kubernetes.io/projected/36fa450d-79ef-416c-b192-c7c43c9d1a15-kube-api-access-hdjkx\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:33:46.111222 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.111230    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36fa450d-79ef-416c-b192-c7c43c9d1a15-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:33:46.485657 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.485626    2567 generic.go:356] "Generic (PLEG): container finished" podID="36fa450d-79ef-416c-b192-c7c43c9d1a15" containerID="2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc" exitCode=0
Sep 18 20:33:46.485851 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.485663    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-988c79b4c-wcwnq" event={"ID":"36fa450d-79ef-416c-b192-c7c43c9d1a15","Type":"ContainerDied","Data":"2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc"}
Sep 18 20:33:46.485851 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.485705    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-988c79b4c-wcwnq"
Sep 18 20:33:46.485851 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.485730    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-988c79b4c-wcwnq" event={"ID":"36fa450d-79ef-416c-b192-c7c43c9d1a15","Type":"ContainerDied","Data":"22d125e8e7a24ab1dcd801fc14f4f11f401b83414fe6bbcb0add14376a5e45fc"}
Sep 18 20:33:46.485851 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.485752    2567 scope.go:117] "RemoveContainer" containerID="2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc"
Sep 18 20:33:46.495212 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.495194    2567 scope.go:117] "RemoveContainer" containerID="2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc"
Sep 18 20:33:46.495474 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:33:46.495454    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc\": container with ID starting with 2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc not found: ID does not exist" containerID="2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc"
Sep 18 20:33:46.495534 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.495483    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc"} err="failed to get container status \"2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc\": rpc error: code = NotFound desc = could not find container \"2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc\": container with ID starting with 2fe203c4e9802997ae014fbb213726297c19df94c60b6f3704acf351232e35cc not found: ID does not exist"
Sep 18 20:33:46.508247 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.508227    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-988c79b4c-wcwnq"]
Sep 18 20:33:46.513568 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:46.513547    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-988c79b4c-wcwnq"]
Sep 18 20:33:47.699015 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:33:47.698981    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36fa450d-79ef-416c-b192-c7c43c9d1a15" path="/var/lib/kubelet/pods/36fa450d-79ef-416c-b192-c7c43c9d1a15/volumes"
Sep 18 20:36:02.090798 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:36:02.090725    2567 scope.go:117] "RemoveContainer" containerID="dcc22cb431bf0491f487a3298513c8064d6916e090dcbacacb8c7a5ede6bdae7"
Sep 18 20:37:01.866865 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:01.866834    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:37:01.875962 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:01.875937    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:37:55.042833 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.042795    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"]
Sep 18 20:37:55.043315 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.043189    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36fa450d-79ef-416c-b192-c7c43c9d1a15" containerName="manager"
Sep 18 20:37:55.043315 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.043200    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="36fa450d-79ef-416c-b192-c7c43c9d1a15" containerName="manager"
Sep 18 20:37:55.043315 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.043294    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="36fa450d-79ef-416c-b192-c7c43c9d1a15" containerName="manager"
Sep 18 20:37:55.046530 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.046511    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.049145 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.049120    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8kcjh\""
Sep 18 20:37:55.065109 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.065082    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"]
Sep 18 20:37:55.156562 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.156515    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b9667d2-5ccc-4405-ba93-473c764aeb64-maas-api-tls\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.156767 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.156586    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b9667d2-5ccc-4405-ba93-473c764aeb64-metrics-certs\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.156767 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.156640    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf5q4\" (UniqueName: \"kubernetes.io/projected/1b9667d2-5ccc-4405-ba93-473c764aeb64-kube-api-access-wf5q4\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.156767 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.156714    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9667d2-5ccc-4405-ba93-473c764aeb64-openshift-service-ca\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.257562 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.257529    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b9667d2-5ccc-4405-ba93-473c764aeb64-maas-api-tls\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.257770 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.257583    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b9667d2-5ccc-4405-ba93-473c764aeb64-metrics-certs\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.257770 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.257618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf5q4\" (UniqueName: \"kubernetes.io/projected/1b9667d2-5ccc-4405-ba93-473c764aeb64-kube-api-access-wf5q4\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.257770 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.257659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9667d2-5ccc-4405-ba93-473c764aeb64-openshift-service-ca\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.258545 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.258517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b9667d2-5ccc-4405-ba93-473c764aeb64-openshift-service-ca\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.260555 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.260527    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b9667d2-5ccc-4405-ba93-473c764aeb64-metrics-certs\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.260675 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.260590    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1b9667d2-5ccc-4405-ba93-473c764aeb64-maas-api-tls\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.271840 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.271811    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf5q4\" (UniqueName: \"kubernetes.io/projected/1b9667d2-5ccc-4405-ba93-473c764aeb64-kube-api-access-wf5q4\") pod \"maas-api-68c86dfd56-zvb5h\" (UID: \"1b9667d2-5ccc-4405-ba93-473c764aeb64\") " pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.356578 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.356538    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:55.643338 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.643307    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"]
Sep 18 20:37:55.644392 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:37:55.644366    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b9667d2_5ccc_4405_ba93_473c764aeb64.slice/crio-8e3a18567062013ad390618d45425d00bf5a2ae2a69c3e2d935181daa9d996ac WatchSource:0}: Error finding container 8e3a18567062013ad390618d45425d00bf5a2ae2a69c3e2d935181daa9d996ac: Status 404 returned error can't find the container with id 8e3a18567062013ad390618d45425d00bf5a2ae2a69c3e2d935181daa9d996ac
Sep 18 20:37:55.731217 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:55.731181    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h" event={"ID":"1b9667d2-5ccc-4405-ba93-473c764aeb64","Type":"ContainerStarted","Data":"8e3a18567062013ad390618d45425d00bf5a2ae2a69c3e2d935181daa9d996ac"}
Sep 18 20:37:56.737611 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:56.737573    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h" event={"ID":"1b9667d2-5ccc-4405-ba93-473c764aeb64","Type":"ContainerStarted","Data":"5a47d8929c1e7437769e1377961be796f0b887081ef51b1375a44d7f5f172863"}
Sep 18 20:37:56.738016 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:56.737650    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:37:56.761424 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:37:56.761369    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h" podStartSLOduration=1.453139099 podStartE2EDuration="1.761351361s" totalImagesPullingTime="308.212262ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:37:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:37:56.759337372 +0000 UTC m=+2457.319920830" watchObservedRunningTime="2026-09-18 20:37:56.761351361 +0000 UTC m=+2457.321934818"
Sep 18 20:38:02.748823 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:02.748790    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68c86dfd56-zvb5h"
Sep 18 20:38:02.815751 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:02.815715    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"]
Sep 18 20:38:02.815964 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:02.815920    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf" podUID="f911de8f-d3db-4261-a12f-3a984f1c2921" containerName="maas-api" containerID="cri-o://28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8" gracePeriod=30
Sep 18 20:38:04.061158 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.061131    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:38:04.149713 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.149580    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-maas-api-tls\") pod \"f911de8f-d3db-4261-a12f-3a984f1c2921\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") "
Sep 18 20:38:04.149713 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.149632    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-metrics-certs\") pod \"f911de8f-d3db-4261-a12f-3a984f1c2921\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") "
Sep 18 20:38:04.149713 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.149667    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911de8f-d3db-4261-a12f-3a984f1c2921-openshift-service-ca\") pod \"f911de8f-d3db-4261-a12f-3a984f1c2921\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") "
Sep 18 20:38:04.150015 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.149763    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rh2w\" (UniqueName: \"kubernetes.io/projected/f911de8f-d3db-4261-a12f-3a984f1c2921-kube-api-access-4rh2w\") pod \"f911de8f-d3db-4261-a12f-3a984f1c2921\" (UID: \"f911de8f-d3db-4261-a12f-3a984f1c2921\") "
Sep 18 20:38:04.150081 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.150022    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f911de8f-d3db-4261-a12f-3a984f1c2921-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f911de8f-d3db-4261-a12f-3a984f1c2921" (UID: "f911de8f-d3db-4261-a12f-3a984f1c2921"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:38:04.151967 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.151940    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f911de8f-d3db-4261-a12f-3a984f1c2921" (UID: "f911de8f-d3db-4261-a12f-3a984f1c2921"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:38:04.152059 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.152003    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f911de8f-d3db-4261-a12f-3a984f1c2921" (UID: "f911de8f-d3db-4261-a12f-3a984f1c2921"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:38:04.152059 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.152023    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f911de8f-d3db-4261-a12f-3a984f1c2921-kube-api-access-4rh2w" (OuterVolumeSpecName: "kube-api-access-4rh2w") pod "f911de8f-d3db-4261-a12f-3a984f1c2921" (UID: "f911de8f-d3db-4261-a12f-3a984f1c2921"). InnerVolumeSpecName "kube-api-access-4rh2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:38:04.250848 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.250814    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rh2w\" (UniqueName: \"kubernetes.io/projected/f911de8f-d3db-4261-a12f-3a984f1c2921-kube-api-access-4rh2w\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:38:04.250848 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.250844    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-maas-api-tls\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:38:04.250848 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.250853    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f911de8f-d3db-4261-a12f-3a984f1c2921-metrics-certs\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:38:04.251080 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.250861    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f911de8f-d3db-4261-a12f-3a984f1c2921-openshift-service-ca\") on node \"ip-10-0-130-248.ec2.internal\" DevicePath \"\""
Sep 18 20:38:04.778476 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.778439    2567 generic.go:356] "Generic (PLEG): container finished" podID="f911de8f-d3db-4261-a12f-3a984f1c2921" containerID="28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8" exitCode=0
Sep 18 20:38:04.778711 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.778487    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"
Sep 18 20:38:04.778711 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.778491    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf" event={"ID":"f911de8f-d3db-4261-a12f-3a984f1c2921","Type":"ContainerDied","Data":"28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8"}
Sep 18 20:38:04.778711 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.778635    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf" event={"ID":"f911de8f-d3db-4261-a12f-3a984f1c2921","Type":"ContainerDied","Data":"7c271cc8bdc6d0a49f00a73d8d830bf7615149118fc41e198531bc03793e25c5"}
Sep 18 20:38:04.778711 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.778669    2567 scope.go:117] "RemoveContainer" containerID="28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8"
Sep 18 20:38:04.788924 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.788903    2567 scope.go:117] "RemoveContainer" containerID="28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8"
Sep 18 20:38:04.789219 ip-10-0-130-248 kubenswrapper[2567]: E0918 20:38:04.789201    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8\": container with ID starting with 28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8 not found: ID does not exist" containerID="28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8"
Sep 18 20:38:04.789266 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.789229    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8"} err="failed to get container status \"28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8\": rpc error: code = NotFound desc = could not find container \"28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8\": container with ID starting with 28227745cb083194cfb1edf1d40bc375f03a409bd405328b536b7a718b5073a8 not found: ID does not exist"
Sep 18 20:38:04.804081 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.804052    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"]
Sep 18 20:38:04.814633 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:04.814608    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-lvzqf"]
Sep 18 20:38:05.698860 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:05.698820    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f911de8f-d3db-4261-a12f-3a984f1c2921" path="/var/lib/kubelet/pods/f911de8f-d3db-4261-a12f-3a984f1c2921/volumes"
Sep 18 20:38:20.873855 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:20.873824    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b6fdfb5bb-n8vdb_1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757/authorino/0.log"
Sep 18 20:38:27.048881 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:27.048851    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f76f79b89-gdn8l_f7909318-b051-4381-81e1-ba519a25722e/manager/0.log"
Sep 18 20:38:27.055725 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:27.055678    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f76f79b89-gdn8l_f7909318-b051-4381-81e1-ba519a25722e/copy-manifests/0.log"
Sep 18 20:38:27.511426 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:27.511398    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-68c86dfd56-zvb5h_1b9667d2-5ccc-4405-ba93-473c764aeb64/maas-api/0.log"
Sep 18 20:38:28.618922 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:28.618897    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b6fdfb5bb-n8vdb_1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757/authorino/0.log"
Sep 18 20:38:30.131565 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:30.131535    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dc98cf6f8-zx5xc_1a313b08-2ac0-4200-ab53-f359a06cb932/kube-auth-proxy/0.log"
Sep 18 20:38:30.373287 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:30.373253    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ktlxz_c0d168c9-2d92-420c-8187-833852d4c767/payload-pre-processing/0.log"
Sep 18 20:38:30.490847 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:30.490766    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xr9rk_056b9973-eb31-492f-b1d2-b2e3877abf2d/payload-processing/0.log"
Sep 18 20:38:31.068877 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:31.068851    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-shtmh_f531fdf6-7de5-4b73-9100-adfefc762b02/main/0.log"
Sep 18 20:38:31.179873 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:31.179848    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-2v8cf_81fbef5d-e2d5-4008-962f-a67dd4cb2985/main/0.log"
Sep 18 20:38:31.532482 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:31.532448    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-j9kj8_10b1bc9e-f555-4e8e-9118-052768bc3111/main/0.log"
Sep 18 20:38:31.649567 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:31.649543    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-fwwmd_a3019236-d88e-48e8-9efb-27eb14631179/main/0.log"
Sep 18 20:38:40.773095 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:40.773062    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-88w6c_2b6d77f4-7f24-4b8e-b5ae-04e68a6c495c/global-pull-secret-syncer/0.log"
Sep 18 20:38:41.103056 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:41.103031    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kd48x_a4a019a5-9f6b-4c63-bd03-a28117dea38e/konnectivity-agent/0.log"
Sep 18 20:38:41.174660 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:41.174626    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-248.ec2.internal_a5775ae8d5cff8302601fb6bb6349719/haproxy/0.log"
Sep 18 20:38:45.699442 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:45.699413    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5b6fdfb5bb-n8vdb_1f5c9f7d-fb9f-43a1-8743-e9d50ad3a757/authorino/0.log"
Sep 18 20:38:47.358136 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.358111    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9fce943-a9ea-4626-8496-f43fbb7787b1/alertmanager/0.log"
Sep 18 20:38:47.391356 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.391329    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9fce943-a9ea-4626-8496-f43fbb7787b1/config-reloader/0.log"
Sep 18 20:38:47.426900 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.426872    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9fce943-a9ea-4626-8496-f43fbb7787b1/kube-rbac-proxy-web/0.log"
Sep 18 20:38:47.454956 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.454936    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9fce943-a9ea-4626-8496-f43fbb7787b1/kube-rbac-proxy/0.log"
Sep 18 20:38:47.493648 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.493618    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9fce943-a9ea-4626-8496-f43fbb7787b1/kube-rbac-proxy-metric/0.log"
Sep 18 20:38:47.528106 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.528066    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9fce943-a9ea-4626-8496-f43fbb7787b1/prom-label-proxy/0.log"
Sep 18 20:38:47.556227 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.556196    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e9fce943-a9ea-4626-8496-f43fbb7787b1/init-config-reloader/0.log"
Sep 18 20:38:47.637369 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.637305    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-whc66_8a18ab65-fbb1-408f-b7f4-9168e5ee4847/kube-state-metrics/0.log"
Sep 18 20:38:47.673531 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.673505    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-whc66_8a18ab65-fbb1-408f-b7f4-9168e5ee4847/kube-rbac-proxy-main/0.log"
Sep 18 20:38:47.707538 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.707498    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-whc66_8a18ab65-fbb1-408f-b7f4-9168e5ee4847/kube-rbac-proxy-self/0.log"
Sep 18 20:38:47.785525 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.785503    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-658b6d7886-v5z8k_7e39f0d9-b2c3-4e34-aa29-e4215056c9cc/metrics-server/0.log"
Sep 18 20:38:47.844077 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:47.844043    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-dgsmk_3e69bd6f-7299-4297-94c1-9e962332bcf2/monitoring-plugin/0.log"
Sep 18 20:38:48.131387 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.131358    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9c582_0a56f260-3fc2-4202-ae5c-86461eb42d71/node-exporter/0.log"
Sep 18 20:38:48.167339 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.167315    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9c582_0a56f260-3fc2-4202-ae5c-86461eb42d71/kube-rbac-proxy/0.log"
Sep 18 20:38:48.189947 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.189924    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9c582_0a56f260-3fc2-4202-ae5c-86461eb42d71/init-textfile/0.log"
Sep 18 20:38:48.348901 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.348868    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed0f345-b86e-42f5-a058-a6fc4398e901/prometheus/0.log"
Sep 18 20:38:48.378876 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.378849    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed0f345-b86e-42f5-a058-a6fc4398e901/config-reloader/0.log"
Sep 18 20:38:48.403064 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.403038    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed0f345-b86e-42f5-a058-a6fc4398e901/thanos-sidecar/0.log"
Sep 18 20:38:48.438099 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.438069    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed0f345-b86e-42f5-a058-a6fc4398e901/kube-rbac-proxy-web/0.log"
Sep 18 20:38:48.471480 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.471454    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed0f345-b86e-42f5-a058-a6fc4398e901/kube-rbac-proxy/0.log"
Sep 18 20:38:48.502067 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.501992    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed0f345-b86e-42f5-a058-a6fc4398e901/kube-rbac-proxy-thanos/0.log"
Sep 18 20:38:48.535844 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.535819    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_0ed0f345-b86e-42f5-a058-a6fc4398e901/init-config-reloader/0.log"
Sep 18 20:38:48.771454 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.771381    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79564b56cd-m2cm4_853ddca3-13e7-4908-925e-7420e61ebd7d/thanos-query/0.log"
Sep 18 20:38:48.795959 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.795934    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79564b56cd-m2cm4_853ddca3-13e7-4908-925e-7420e61ebd7d/kube-rbac-proxy-web/0.log"
Sep 18 20:38:48.820254 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.820230    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79564b56cd-m2cm4_853ddca3-13e7-4908-925e-7420e61ebd7d/kube-rbac-proxy/0.log"
Sep 18 20:38:48.842945 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.842919    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79564b56cd-m2cm4_853ddca3-13e7-4908-925e-7420e61ebd7d/prom-label-proxy/0.log"
Sep 18 20:38:48.867398 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.867375    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79564b56cd-m2cm4_853ddca3-13e7-4908-925e-7420e61ebd7d/kube-rbac-proxy-rules/0.log"
Sep 18 20:38:48.901547 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:48.901523    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-79564b56cd-m2cm4_853ddca3-13e7-4908-925e-7420e61ebd7d/kube-rbac-proxy-metrics/0.log"
Sep 18 20:38:50.286995 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.286970    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-44k5c_c6ad647b-347c-4d77-8a59-4420270b4209/networking-console-plugin/0.log"
Sep 18 20:38:50.532227 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.532188    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"]
Sep 18 20:38:50.532597 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.532585    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f911de8f-d3db-4261-a12f-3a984f1c2921" containerName="maas-api"
Sep 18 20:38:50.532641 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.532599    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f911de8f-d3db-4261-a12f-3a984f1c2921" containerName="maas-api"
Sep 18 20:38:50.532680 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.532669    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f911de8f-d3db-4261-a12f-3a984f1c2921" containerName="maas-api"
Sep 18 20:38:50.536068 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.536046    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.541834 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.541769    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w4xpn\"/\"openshift-service-ca.crt\""
Sep 18 20:38:50.541834 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.541809    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w4xpn\"/\"default-dockercfg-68gcr\""
Sep 18 20:38:50.542544 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.542470    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w4xpn\"/\"kube-root-ca.crt\""
Sep 18 20:38:50.555068 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.555041    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"]
Sep 18 20:38:50.720195 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.720160    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsbvt\" (UniqueName: \"kubernetes.io/projected/f56f53b5-c2ab-4319-85e7-a79f3cab6237-kube-api-access-lsbvt\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.720399 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.720243    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-sys\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.720399 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.720276    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-lib-modules\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.720399 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.720322    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-proc\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.720399 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.720383    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-podres\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.821470 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.821380    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-proc\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.821470 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.821433    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-podres\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.821653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.821477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsbvt\" (UniqueName: \"kubernetes.io/projected/f56f53b5-c2ab-4319-85e7-a79f3cab6237-kube-api-access-lsbvt\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.821653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.821486    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-proc\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.821653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.821539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-sys\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.821653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.821560    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-lib-modules\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.821653 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.821641    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-sys\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.821882 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.821717    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-podres\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.821882 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.821751    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f56f53b5-c2ab-4319-85e7-a79f3cab6237-lib-modules\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.832022 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.831996    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsbvt\" (UniqueName: \"kubernetes.io/projected/f56f53b5-c2ab-4319-85e7-a79f3cab6237-kube-api-access-lsbvt\") pod \"perf-node-gather-daemonset-xcbks\" (UID: \"f56f53b5-c2ab-4319-85e7-a79f3cab6237\") " pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.845573 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.845547    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:50.980723 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.980696    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/1.log"
Sep 18 20:38:50.985788 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:50.985764    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-f7n5c_6ef6ccf3-1c81-4c80-a3f4-1e85d49ea4da/console-operator/2.log"
Sep 18 20:38:51.120579 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:51.120555    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"]
Sep 18 20:38:51.122853 ip-10-0-130-248 kubenswrapper[2567]: W0918 20:38:51.122827    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf56f53b5_c2ab_4319_85e7_a79f3cab6237.slice/crio-78631088f58fd00ba1a5fce796fa65130c206c6748ad54843d8335e1e05c3f56 WatchSource:0}: Error finding container 78631088f58fd00ba1a5fce796fa65130c206c6748ad54843d8335e1e05c3f56: Status 404 returned error can't find the container with id 78631088f58fd00ba1a5fce796fa65130c206c6748ad54843d8335e1e05c3f56
Sep 18 20:38:51.578547 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:51.578459    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-59d7cd6bb6-rspnq_2561e8ac-3e9f-49b0-820f-ad7fb0eaa1f4/console/0.log"
Sep 18 20:38:52.020454 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:52.020404    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks" event={"ID":"f56f53b5-c2ab-4319-85e7-a79f3cab6237","Type":"ContainerStarted","Data":"3b30ac8c2d67c2283754abf288ce1a0a5b039fbc45e39f652d4592fcb4c76e2a"}
Sep 18 20:38:52.020654 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:52.020465    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks" event={"ID":"f56f53b5-c2ab-4319-85e7-a79f3cab6237","Type":"ContainerStarted","Data":"78631088f58fd00ba1a5fce796fa65130c206c6748ad54843d8335e1e05c3f56"}
Sep 18 20:38:52.020654 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:52.020497    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:52.050729 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:52.050663    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks" podStartSLOduration=2.050648072 podStartE2EDuration="2.050648072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:38:52.049209912 +0000 UTC m=+2512.609793369" watchObservedRunningTime="2026-09-18 20:38:52.050648072 +0000 UTC m=+2512.611231571"
Sep 18 20:38:53.821841 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:53.821808    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wtp9b_076b49bd-a8a9-4595-8a08-fedc2b71d2ea/dns/0.log"
Sep 18 20:38:53.846645 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:53.846620    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wtp9b_076b49bd-a8a9-4595-8a08-fedc2b71d2ea/kube-rbac-proxy/0.log"
Sep 18 20:38:53.879155 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:53.879132    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5pgn9_79818bbf-4ed9-4ffe-ace4-dd7492917365/dns-node-resolver/0.log"
Sep 18 20:38:54.501954 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:54.501927    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6678b7486f-zmglw_44092adf-564e-4dfa-b9d7-1a9dc81417a1/registry/0.log"
Sep 18 20:38:54.592414 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:54.592383    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sfbf2_adcd525f-a8d8-4903-95d6-37a8b8397f60/node-ca/0.log"
Sep 18 20:38:56.347943 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:56.347914    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dc98cf6f8-zx5xc_1a313b08-2ac0-4200-ab53-f359a06cb932/kube-auth-proxy/0.log"
Sep 18 20:38:56.429080 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:56.429039    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ktlxz_c0d168c9-2d92-420c-8187-833852d4c767/payload-pre-processing/0.log"
Sep 18 20:38:56.467459 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:56.467432    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xr9rk_056b9973-eb31-492f-b1d2-b2e3877abf2d/payload-processing/0.log"
Sep 18 20:38:57.825270 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:57.825235    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-k4kgb_4fb83fb2-b7ef-4134-8f0b-556d934f2359/serve-healthcheck-canary/0.log"
Sep 18 20:38:58.036469 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:58.036442    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w4xpn/perf-node-gather-daemonset-xcbks"
Sep 18 20:38:58.449863 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:58.449835    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f9c2r_5ba68aef-6f7a-4620-8af1-092ce821ea07/kube-rbac-proxy/0.log"
Sep 18 20:38:58.474077 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:58.474050    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f9c2r_5ba68aef-6f7a-4620-8af1-092ce821ea07/exporter/0.log"
Sep 18 20:38:58.510433 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:38:58.510405    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f9c2r_5ba68aef-6f7a-4620-8af1-092ce821ea07/extractor/0.log"
Sep 18 20:39:01.619833 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:01.619805    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dc98cf6f8-zx5xc_1a313b08-2ac0-4200-ab53-f359a06cb932/kube-auth-proxy/0.log"
Sep 18 20:39:01.691984 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:01.691957    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ktlxz_c0d168c9-2d92-420c-8187-833852d4c767/payload-pre-processing/0.log"
Sep 18 20:39:01.736275 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:01.736229    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xr9rk_056b9973-eb31-492f-b1d2-b2e3877abf2d/payload-processing/0.log"
Sep 18 20:39:02.044004 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:02.043919    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f76f79b89-gdn8l_f7909318-b051-4381-81e1-ba519a25722e/manager/0.log"
Sep 18 20:39:02.094521 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:02.094489    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-f76f79b89-gdn8l_f7909318-b051-4381-81e1-ba519a25722e/copy-manifests/0.log"
Sep 18 20:39:03.384450 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:03.384404    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5848b5686c-hmtqx_216ce354-df51-4524-9aac-0258d59c7948/manager/0.log"
Sep 18 20:39:03.414731 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:03.414706    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zjzxj_dc2cd8f0-bf11-40d2-a9d3-d8c48411944e/openshift-lws-operator/0.log"
Sep 18 20:39:04.228376 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:04.228334    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dc98cf6f8-zx5xc_1a313b08-2ac0-4200-ab53-f359a06cb932/kube-auth-proxy/0.log"
Sep 18 20:39:04.301484 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:04.301455    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-ktlxz_c0d168c9-2d92-420c-8187-833852d4c767/payload-pre-processing/0.log"
Sep 18 20:39:04.333346 ip-10-0-130-248 kubenswrapper[2567]: I0918 20:39:04.333302    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-xr9rk_056b9973-eb31-492f-b1d2-b2e3877abf2d/payload-processing/0.log"